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Preface 


This volume contains ten chapters on numerical methods which could be 
gainfully employed by scientists and engineers to solve the problems arising 
in research and industry. It also covers the syllabus prescribed for engmeering 
studies at the undergraduate level. An essential feature of the present edition 
is that it provides information about the readily available computer program 
packages for implementing the numerical methods described in the book. 
These include references to MATLAB, IMSL and Numerical Recipes program 
libraries. Several problems have been set as exercises to illustrate the use 
of these libraries. Nevertheless, for a better understanding of these methods, 
the readers are advised to develop their own programs in any computer 
language of their choice. 

More than two decades have elapsed since the first appearance of this 
book and, quite naturally, there have been many changes in the presentation 
of the material as also new additions of topics to meet the changing 
requirements of students in various universities. Thus, topics like curve 
fitting procedures, cubic spline methods, approximation of functions, numerical 
solution of integral equations, Graeffe’s root-squaring method, weighted 
least-squares approximations, B-splines, Householder and QR methods, singular 
value decomposition, shooting method, the ADI method and the finite element 
method were gradually added to enhance the utility of the book. In the 
present edition, most sections have been rewritten to provide a better 
understanding of the topics. Thus the section on cubic splines has been 
rewritten with the inclusion of linear and quadratic splines, and a new 
section on surface fitting by cubic splines has been added. Similarly, a new 
section on Fourier transforms has also been included. The worked examples 
have been modified, new problems have been introduced and the number of 
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worked examples and homework problems has been significantly increased. 
This edition therefore contains about 500 problems including the illustrative 
examples and exercises for homework. Answers have been provided to 
some selected end-of-chapter exercises. 

The author is very much obliged to the students and teachers of various 
universities who have been using this book during the last several years. 
Grateful thanks are due to Prof. I. Chandra Mohan, Sri Venkateswara University, 
Tirupati, for his suggestion to derive the general formulae in predictor— 
corrector methods in Chapter 7. Any suggestions towards the improve- 
ment of the book will be gratefully accepted. Special thanks are due to 
Sn Asoke K. Ghosh, Chairman and Managing Director, Prentice-Hall of India, 
New Delhi, for his courteous cooperation in bringing out this edition. 


Chennai 8.5. SASTRY 
February, 2005 
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Errors in Numerical Calculations 


1.1 INTRODUCTION 


In practical applications, an engineer would finally obtain results in a numerical 
form. For example, from a set of tabulated data derived from an experiment, 
inferences may have to be drawn; or, a system of linear algebraic equations. 
is to be solved, The aim of numerical analysis is to provide efficient methods 
for obtaining numerical answers to such problems. This book deals with 
methods of numerical analysis rather than the analysis of numerical methods, 
because our main concern is to provide computer-oriented, efficient and 
reliable numerical methods for solving problems arising in different areas of 
higher mathematics. The areas of numerical mathematics, addressed in this 
book, are: 


(a) Algebraic and transcendental equations: The problem of solving 


(b) 


nonlinear equations of the type f(x)=0 is frequently encountered 
in engineering. For example, the equation 


— Mo _ keratin (1.1) 
My = Myf 
is a nonlinear equation for tf when My, g,u, ug and wy are given. 
Equations of this type occur in rocket studies. 
Jnterpolation: Given a set of data values (x, ¥,), (= 0,1,2,...,98, of 
a function y= f(x), where the explicit nature of f(x) is not known, 
it is often required to find the value of y for a given value of x, 
where x, <x<x,. This process is called interpolation. If this process 
1 


Presented By: http://www.ebooksuit.com 


2 CuHaprer 1+ Errors in Numerical Calculations 


is carried out for functions of several variables, it is called mud/tivariate 
interpolation. 


(c) Curve fitting: This is a special case where the data points are 
subject to errors, both round off and systematic. In such a case, 
interpolation formulae yield unsatisfactory solutions. Experimental 
results are often subject to errors and, in such cases, the method 
is to fit a curve which passes through the data points and then use 
the curve to predict the intermediate values. This problem is usually 
referred to as data smoothing. 

(d) Numerical differentiation and integration: It is often required to 
determine the numerical values of 


2 
(i) *. ope for a certain value of x in xy <x<x,, and 


(ii) [= |™ y de, 


where the set of data values (x, ¥,), 7=0,1,...,”2 is given, but the 
explicit nature of (x) is not known, For example, if the data consist 
of the angle @ (in radians) of a rotating rod for values of time f (in 
seconds), then its angular velocity and angular acceleration at any 
time can be computed by numerical differentiation formulae. 


(e) Matrices and linear systems: The problem of solving systems of 
linear algebraic equations and the determination of eigenvalues and 
eigenvectors of matrices are major problems of disciplines such as 
differential equations, fluid mechanics, theory of structures, etc. 


(f) Ordinary and partial differential equations: Engineering problems 
are often formulated in terms of an ordinary or a partial differential 
equation. For example, the mathematical formulation of a falling 
body involves an ordinary differential equation and the problem of 
determining the steady-state distribution of temperature on a heated 
plate is formulated in terms of a partial differential equation. In 
most cases, exact solutions are not possible and a numerical method 
has to be adopted. In addition to the finite difference methods, this 
book also presents a brief introduction to the finite element method 
for solving partial differential equations. 


(g) Jntegral equations; An equation in which the unknown function 
appears under the integral sign is known as an integral equation. 
Equations of this type occur in several areas of higher mathematics 
such as aerodynamics, elasticity, electrostatics, etc. A short account 
of some well-known methods 1s given. 

In the numerical solution of problems, we usually start with some 
initial data and then compute, after some intermediate steps, the 
final results. The given numerical data are only approximate because 
they may be true to two, three or more figures. In addition, the 
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methods used may also be approximate and therefore the error in 
a computed result may be due to the errors in the data, or the 
errors in the method, or both. In Section 1.3, we discuss some 
basic ideas concerning errors and their analyses, since such an 
understanding is essential for an effective use of numerical methods. 
Before discussing about errors in computations, we shall first look 
into some important computer languages and software. 


1.1.1. Computer and Numerical Software 


It is well known that computers and mathematics are two important tools 
of numerical methods, Prior to 1950, numerical methods could only be 
implemented by manual computations, but the rapid technological advances 
resulted in the production of computing machines which are faster, economical 
and smaller in size. Today’s engineers have access to several types of 
computing systems, viz., mainframe computers, personal computers and 
super computers. Of these, the personal computer is a smaller machine 
which is useful, less expensive and, as the name implies, can easily be 
possessed and used by individuals. Nevertheless, mere possession of a 
computer is not of great consequence; it can be used effectively only by 
providing suitable instructions to it. These instructions are known as software. 
It is therefore imperative that we develop suitable software for an effective 
implementation of numerical methods on computers. 

Essentially, there are three phases in the development of numerical 
software for solving a problem. In the first phase, the problem to be solved 
must be formulated mathematically indicating the input and outputs and also 
the checks to be made on the solution. The second phase consists of 
choosing an algorithm, i.e., a suitable numerical procedure to solve the 
mathematical problem. An algorithm is a set of instructions leading to the 
solution of the mathematical problem, and also contains information regarding 
the accuracy required and computation of error in the solution. In the final 
phase, the algorithm must be transformed into a computer program (called 
code) which is a set of step-by-step instructions to the computer written 
in a computer language. Usually, it may be preferable to prepare a flowchart 
first and then transform the flowchart into a computer program. The flowchart 
consists of the step-by-step procedures, in block form, which the computer 
will follow and which can easily be understood by others who wish to 
know about the program. It is easy to see that the flowchart enables a 
programmer to develop a quality computer program using one of the computer 
languages listed in the next section. However, experienced programmers 
often transform a detailed algorithm into an efficient computer program. 


1.1.2 Computer Languages 


Several computer languages have so far been developed and there are limitations 
on every language. The question of preferring a particular ‘language over 
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others depends on the problem and its requirements. We list below some 
important problem-solving languages, which are currently in use: 


(a) FORTRAN: Standing for FORmula TRANslation, FORTRAN was 
introduced by IBM in 1957. Since then, it has undergone many 
changes and the present version, called FORTRAN 90, is favoured 
by most scientists and engineers. [t is readily available on almost 
all computers and one of its important features is that it allows a 
programmer to express the mathematical algorithm more precisely. It 
has special features like extended double precision, special mathematical 
functions and complex variables. Besides, FORTRAN is the language 
used in numerically oriented subprograms developed by many software 
libraries. For example, (IMSL) (International Mathematical and 
Statistical Library, Inc.) consists of FORTRAN subroutines and 
functions in applied mathematics, Statistics and special functions. 
FORTRAN programs are also available in the book, Numerical Recipes, 
published by the Cambridge University Press, for most of the standard 
numerical methods. 

(b) C: This is a high-level programming language developed by Bell 
Telephone Laboratories in 1972. Presently, it is being taught at 
several engineering colleges as the first computer language and is 
therefore used by a large number of engineers and scientists. Computer 
programs in C for standard numerical methods are available in the 
book, Numerical Recipes in C, published by the Cambridge University 
Press. 


(c) BASIC: Originally developed by John Kemeny and Thomas Kurtz 
in 1960, BASIC was used in the first few years only for instruction 
purposes. Over the years, it has grown tremendously and the present 
version 1s called Visual Basic. One of its important applications is 
in the development of software on personal computers. It is easy 
to use. 


1.1.3 Software Packages 


It is well known that the programming effort is considerably reduced by 
using standard functions and subroutines. Several software packages for 
numerical methods are available in the form of ‘functions’ and these are 
being extensively used by engineering students. One such package is MATLAB, 
standing for MATrices LABoratory. It was developed by Cleve Moler and John 
N. Little. As the name implies, it was originally founded to develop a matrix 
package but now it incorporates several numerical methods such as root- 
finding of polynomials, cubic spline interpolation, discrete Fourier transforms, 
numerical differentiation and integration, ordinary differential equations and 
eigenvalue problems. Besides, MATLAB has excellent display capabilities 
which can be used in the case of two-dimensional problems. Using the 
MATLAB functions, it is possible to implement most of the numerical methods 
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on personal computers and hence it has become one of the most 
popular packages in most laboratories and technical colleges. MATLAB has 
its own programming language and this is described in detail in the text by 
Stephen J. Chapman.* 


1.2 MATHEMATICAL PRELIMINARIES 


In this section we state, without proof, certain mathematical results which 
would be useful in the sequel. 


Theorem 1.1 \f f(x) is continuous in a=x<h, and if f(a) and f(4) are 
of opposite signs, then f(£)=0 for at least one number € such that a< é <4, 


Theorem 1.2 (Rolle’s thearem) If f(x) is continuous in a= x<h, f'(x) 
exists ina<x<b and f(a)= /(5)=0, then, there exists at least one value 
of x, say é, such that f‘(¢)=0, a<G <5, 


Theorem 1.3 (Generalized Rolle’s theorem) Let f(x) be a function which 

is m times differentiable on [a, 6]. If f(x) vanishes at the (m+ 1) distinct 

points x9, %;,....x, in (a, 46), then there exists a number € in (a, 5) such that 
her, — 

FE) =0. 


Theorem 1.4 (intermediate value theorem) Let f(x) be continuous in 
[a, b] and let & be any number between f(a) and f(b). Then there exists 
a number & in (a, 6) such that f(¢)=4 (see Fig. 1.1). 


a E b 
Figure 1.1 


Theorem 1.8  (Mean-value theorem for derivatives) If f(x) is continuous 
in [a, 6] and /'(x) exists in (a, 6), then there exists at least one value of 
x, say €, between a and 4 such that 


f(é)= eee =iett a<f<b, 


*Published by Thomson Asia Pte. Ltd., Singapore (2002). 
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Setting b=a+A, this theorem takes the form 
Flat+hy= f(a)+ hf'(a+@h), O<@<1, 


Theorem 1.6 (Taylor's series for a function of one variable) Uf f(x) is 
continuous and possesses continuous derivatives of order n in an interval 
that includes x =a, then in that interval 


a =o) are a FOP (a)+ Ry (2), 


where A,(x), the remainder term, can be expressed in the form 


F (x)= f(a)+ (x - a) f(a) +———_ 


R(s)= F=f, ac bx. 


Theorem 1.7 (Maclaurin's expansion) It states 


f(x) = FO} +f 0) +5, : (Ff O+- +e i) 


Theorem 1.8 (Taylor's series for a function of two variables) It states 


F(x, + Ax, x2 + Ax) = f(si.%2) +2 eon 


This can easily be generalized. 
Theorem 1.9 (Taylor's series for a function of several variables) 
ty + Ax), X3 + AX5,. 065%, + Ax,,) 


=A Ravnky) + SO + Zon, tt Ay 


1} a a af 
+4|2har tt 5 Ory) 428 ar, Ax, Ax +++ 
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1.3 ERRORS AND THEIR COMPUTATIONS 


There are two kinds of numbers, exact and approximate numbers. Examples 
of exact numbers are 1, 2,3, ..., 1/2, 3/2, saga yt é, etc., written in this 
manner. Approximate numbers are those that represent the numbers to a 
certain degree of accuracy. Thus, an approximate value of w is 3.1416, or 
if we desire a better approximation, it is 3.14159265. But we cannot write 
the exact value of 7. 

The digits that are used to express a number are called significant digits 
or significant figures. Thus, the numbers 3.1416, 0.66667 and 4.0687 contain 
five significant digits each. The number 0.00023 has, however, only two 
significant digits, viz., 2 and 3, since the zeros serve only to fix the position 
of the decimal point. Similarly, the numbers 0.00145, 0.000145 and 0.0000145 all 
have three significant digits. In case of ambiguity, the scientific notation should 
be used. For example, in the number 25,600, the number of significant figures 
is uncertain, whereas the numbers 2.56 x10", 2.56010" and 2.5600 10° 
have three, four and five significant digits, respectively. 

In numerical computations, we come across numbers which have large 
number of digits and it will be necessary to cut them to a usuable number of 
figures. This process is called rounding off. It is usual to round-off numbers 
according to the following rule: 

To round-off a number to » significant digits, discard all digits 
to the right of the nth digit, and if this discarded number ts 


(a) Jess than half a unit in the mth place, leave the ath digit unaltered; 

(b) greater than half a unit in the mth place, increase the nth digit by 
unity; 

(c) exactly half a unit in the mth place, increase the mth digit by unity 
if it is odd; otherwise, leave it unchanged. 


The number thus rounded-off is said to be correct to nm significant 
figures. 


Example 1.1 The numbers given below are rounded-off to four significant 
figures: 


1.6583 to 1.658 
30.0567 to 30.06 

0.859378 to 0.8594 

3.14159 to 3.142 


In hand computations, the round-off error can be reduced by carrying 
out the computations to more significant figures at each step of the 
computation. A useful rule is: at each step of the computation, retain at least 
one more significant figure than that given in the data, perform the last 
operation and then round-off. However, most computers allow more number 
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of significant figures than are usually required in engineering computations. 
Thus, there are computers which allow a precision of seven significant 
figures in the range of about 10°°* to 10°°. Arithmetic carried out with this 
precision is called single precision arithmetic, and several computers implement 
double precision arithmetic, which could be used in problems requiring 
greater accuracy. Usually, the double precision arithmetic is carried out to 
15 decimals with a range of about 107°" to 10°"*. In MATLAB, there is a 
provision to use double precision arithmetic. 

In addition to the round-off error discussed above, there is another type 
of error which can be caused by using approximate formulae in computations, 
—such as the one that arises when a fruncated infinite series is used. This 
type of error is called iruncation error and its study is naturally associated 
with the problem of convergence. Truncation error in a problem can be 
evaluated and we are often required to make it as small as possible. Sections |.4 
and 1.5 will be devoted to a discussion of these errors. 


Absolute, relative and percentage Errors 


Absolute error is the numerical difference between the true value of a 
quantity and its approximate value. Thus, if X is the true value of a quantity 
and .’, is its approximate value, then the absolute error E, is given by 


E,=X -X, =8X. (1.2) 
The relative error Ep, is defined by 
E, 6X 
| el (1.3) 
ae ae 


and the percentage error (Ep) by 


Let AY be a number such that 
|X, -X|sAX. (1.5) 


Then AX is an upper limit on the magnitude of the absolute error and is 
said to measure absolute accuracy, Similarly, the quantity 


AX At 
IX] 1%] 
measures the relative accuracy. 
It is easy to deduce that if two numbers are added or subtracted, then 
the magnitude of the absolute error in the result is the sum of the magnitudes 
of the absolute errors in the two numbers. More generally, if E! re oes A 


are the absolute errors in n numbers, then the magnitude of the absolute 
error in their sum is given by 


[EL I+1 ES | +. +) ES I, 
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Note: While adding up several numbers of different absolute accuracies, the 
following procedure may be adopted: 


(i) Isolate the number with the greatest absolute error, 


(ii) Round-off all other numbers retaining in them one digit more than 
in the isolated number, 


(iii) Add up, and 
(iv) Round-off the sum by discarding one digit. 


To find the absolute error, E,, in a product of two numbers @ and 4, 
we write E, =(a+E,)(b+E%)~-ab, where E\ and EX are the absolute 
errors in @ and 4 respectively. Thus, 


E,=aE, +bE, +E\E, 
=bE} +aE 4, approximately (1.6) 
Similarly, the absolute error in the quotient a/b is given by 
a+E} a_ bE} -aE; 


b+E2 b b(b+E?) 


bE - aE} 

~ b+ E2/b) 
bE, ~aE 4 Za: . : 

= — assuming that £} /b is small in comparison with | 
b| @ b 


Example 1.2 If the number X is rounded to N decimal places, then 
ee 
AY=—(10'"). 
3 ) 
If ¥ =0.51 and is correct to 2 decimal places, then AY = 0.005, and the 
relative accuracy is given by 0.005/0.51= 0.98%. 


Example 1.3 An approximate value of ¢ is given by X, =22/7=3.1428571 
and its true value is WY =3.1415926. Find the absolute and relative errors. 
We have 


E, =X — X, = -0.0012645 
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and 


—0.0012645 


= ————— = -0,000402. 
R35, 1415926 


Example [.4 Three approximate values of the number 1/3 are given as 
0.30, 0.33 and 0.34. Which of these three is the best approximation? We have 


1 930 me 

3 30 

4 -o33]-201 
3 300 

1-04] 202 1 
3 150 


It follows that 0.33 is the best approximation for 1/3. 


Example 1.5 Find the relative error of the number 8.6 if both of its digits 
are correct. 


Here 
E, = 0.05 
Hence 
0.05 
Ey = —— = 0.0058. 
onme T 


Example 1.6 Evaluate the sum S=,/3+./5+,/7 to 4 significant digits 
and find its absolute and relative errors. 


We have 
3 =1.732, {5=2.236 and {7 =2.646 
Hence 5 = 6.614. Then 


£, = 0.0005 + 0.0005 + 0.0005 = 0.0015 


The total absolute error shows that the sum is correct to 3 significant 
figures only. Hence we take S= 6.61 and then 


0.0015 
rey 
Example 1.7 Sum the following numbers: 
0.1532, 15.45, 0.000354, 305.1, 8.12, 143.3, 0.0212, 0.643 and 0.1734. 
where in each of which all the given digits are correct. 


= 0.0002. 


Here we have two numbers which have the greatest absolute error. 
These are 305.1 and 143.3 and the absolute error in both these is 0.05. 
Hence, we round-off all the other number to two decimal digits. These are: 
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0.15, 15.45, 0.00, 8.12, 0.02, 0.64 and 0.17. 
The Sum S§ is given by 
S$ =305.14 143.34 0.154+15.454 0.00 + 8.12 + 0.02 + 0.644 0.17 


= 472,59 
= 472.6 


To determine the absolute error, we note that the first-two numbers have 
each an absolute error of 0.05 and the remaining seven numbers have an 
absolute error of 0.005 each. Thus the absolute error in all the 9 numbers is 


Ey = 2(0.05)+7(0.005) 
= 0.140.035 
= 0.135 
=0.14 


In addition to the above absolute error, we have to take into account the 
rounding error in the above and this is 0.01. Hence the total absolute error 
is § =0.144+0.01=0.15, Thus, 


5 =472.64 0.15. 
Example 1.8 Two numbers are given as 2.5 and 48.289, both of which 
being correct to the significant figures given, Find their product, 


Here the number with the greatest absolute error is 2.5. Hence we 
round-off the second number to three significant digits, ic. 48.3. Their 
product is given by 

P = 48.3x 2.5 =120.75 = 1.2107, 


where we have retained only two significant digits since one of the given 
numbers, viz., 2.5, contained only two significant digits. 


1.4 A GENERAL ERROR FORMULA 


In this section, we derive a general formula for the error committed in using 
a certain formula or a functional relation. Let 


w= f(x, Xp,-..5%_) (1.8) 


be a function of several variables x, (i =1,2,...,.), and let the error in each 
x, be Ax,. Then the error Aw in u is given by 


ut Au = f(x, + AX), Xp + Axy,...,X, + AX, ). (1.9) 


Expanding the right-hand-side by Taylor’s series (see Theorem 1.9), we 
obtain 
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A 
ut Au = f(x), XpsecnX m+d, on + terms involving (Ax,)*. (1.10) 


ne 


Assuming that the errors in x, are small and that (Ax,)/x, <<1, so that 
the squares and higher powers of Ax, can be neglected, the above relation 
yields 


— of of of af 
Aus ¥) Ax, =~ Ax +—— Ax, +--+ Ax, (1.11) 
z 2 a cy |) a 7 ax 


Ht 


We observe that this formula has the same form as that for the total differential 
of uw. The formula for the relative error follows immediately: 


ig ee eB, (1.12) 
if an; Pty Oks u OX, u 
The following example illustrates the use of this formula. 


Example 1.9 Let u = 5xy" lz, 


Then 
au Sy du _1xy u__ Sx? 
ox Pa y 2 : a zt 
and 
Sy? 10xy , 5x" 
Au =——Axr+—= Ay- Az 
2 z . 


In general, the errors Ax, Ay and Az may be positive or negative, and 
hence we take the absolute values of the terms on the right side. This gives 


y 


2 
107 4, = Az\. 


(At) sax * - 


ee 


Now, let Ax = Ay = Az = 0.001 and x= y=z=1. Then, the relative maximum 
error (Ep)max iS given by 


1.5 ERROR IN A SERIES APPROXIMATION 
The truncation error committed in a series approximation can be evaluated 


by using Taylor’s series stated in Theorem 1.6. If x; and x;,, are two 
successive values of x, then we have 
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Fan=SO)+ Gat ADS Cat AA (3) 4 Ryaa Siar) (1.13) 


where 


Rest X41) = SELL" pened gy eee (1.14) 
(n+l)! " 

In (1.13), the last term, R,,,(2,)), is called the remainder term which, 
for a convergent series, tends to zero as n-+o, Thus, if f(x;,)) is 
approximated by the first-n terms of the series (1.13), then the maximum error 
committed by using this approximation (called the nth order approximation) 
is given by the remainder term &,,,,(x,,,). Conversely, if the accuracy required 
is specified in advance, then it would be possible to find m, the number of 
terms, such that the finite series yields the required accuracy. 

Defining the interval length 


Xie Xj =A, (1.15) 
Eq. (1.13) may be written as 


Zz rt 
P41) = Fa) + G)+ SI") toe FC) +O, (1.16) 


where O(f"*') means that the truncation error is of the order of A”*', ice. 
it is proportional to A”*!, The meaning of this statement will be made clearer 
now. 

Let the series be truncated after the first term. This gives the zero-order 
approximation: 


Fj.) = f+ OA), (1.17) 
which means that halving the interval length A will also halve the error in 
the approximate solution. Similarly, the first-order Taylor series approximation 
is given by 

Fj) = fy) + Af") + O17), (1.18) 


which means that halving the interval length, h will quarter the error in the 
approximation. In such a case we say that approximation has a second-order 
of convergence. We illustrate these facts through numerical examples. 


Example 1.10 Evalute f(1) using Taylor’s series for f(x), where 
f(xpox -3x? +5x~10, 


It is easily seen that f(1)=—7 but it will be instructive to see how the 
Taylor series approximations of orders 0 to 3 improve the accuracy of f(1) 


gradually. 
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Let h=1, x, =0 and x,,,; =1. We then require f(x,,,). The derivatives 


of f(x) are given by 
f'(x) =3x" -6x+5, f"(x)=6x-6, f(x) =6, 
f(x) and higher derivatives being all zero. Hence 
I(x) = f'O) =5, Fe) = fF") = +6, f'"(0) = 6. 
Also, 
F(%) = f()=-10. 
Hence, Taylor's series gives 


2 
I (Xia) = FOR) + AP) + =f" ) py ). 


From (i), the zero-order approximation is given by 
F(%ja1) = £0) + OCA), 
and therefore 
F() = FO) + O(A) = -10, 


the error in which is -7+10, i.e. 3 units. 


For the first approximation, we have 
FX) =F) + Af) +00"), 
and therefore 


f() =-10 +540(h*) =-5, 
the error in which is -7+5, i.e. —2 units. 
Again, the second-order Taylor approximation is given by 
2 
Fina) = SO) +H G+ FG) 4000), 
and therefore 


1) =-10+5+5(-6) +O(h*) = -8, 


in which the error is -—7 +8, i.e. 1 unit. 
Finally, the third-order Taylor series approximation is given by 


fed=fadeM aE san+£ rep, 
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and therefore 


hr 


2 
fQ) = f(O)+ Wf (ap) += f"C%)4 =i 


| l 
—10+5+—(-6) +-(6 
= + ae 2) 


=—7, 


which is the exact value of f(1). 

This example demonstrates that if the given function is a polynomial of 
third degree, then its third-order Taylor series approximation gives exact 
results. 


Example 1.11 Given f(x)=sin x, construct the Taylor series approximations 
of orders 0 to 7 at x= 7/3 and state their absolute errors. 


Let x,,,; =2/3 and x, =2/6 so that h=2/3-2/6=2/6. We then have 


SAEs 
Erk @rderGon 


a )sfs) Els) HeH (md) led 
ale) (5) nals) (os) aS) (5) 
“soale) (4) 


0542 pte _W3(a) , U(x) V3 (a) ! (sy 


12 436 12\6) 48.6) 24016) 144016 


The different orders of approximation can now be evaluated successively. 
Thus, the zero-order approximation is 0.5; the first-order approximation is 
0.5+,/3/12, i.e. 0.953449841; and the second-order approximation is 
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which simplifies to 0.884910921, Similarly, the successive approximations 
are evaluated and the respective absolute errors can be calculated since the 
exact value of sin (#/3) is 0.866025403. Table 1.1 gives the approximate 
values of sin (2/3) for the orders 0 to 7 as also the absolute errors in these 
approximations, The results show that the error decreases with an increase 
in the order of approximation. 


Table 1.1. Taylor Series Approximations of f(x) =sin x 


Order of approximation Computed value of sin «/3 Absolute error 

0 0.5 0.366025403 
1 0.953449841 O.087424435 
2 0,884910021 0.076885518 
3 0.664191613 0.00183379 

4 0.865 757474 0.000267929 
5 0.86604149 0.00001 6087 
6 0.66602716 0.000001 777 
7 0.866025326 0.000000077 


We next demonstrate the effect of halving the interval length on any approximate 
value. For this, we consider the first-order approximation in the form: 


F(e+h)= f(x) + Af) + EA), (ii) 
where £(/) is the absolute error of the first-order approximation with interval 
h, Taking f(x)=sin x and x=.2/6, we obtain 

in| ~+h|=sin= = iti 
sin( £8] sin = +h cos = + E(h) (iil) 
Putting A= 7/6 in (iii), we get 


sin? = =0.5+—— a3 —— + E(A) = 0.95344984 1 + E(h). 


Since sin (7/3) aauesia: the above equation gives 
E(h) = -0.087424438. 
Now, let the interval be halved so that we now take A= 2/12. Then, (iii) gives: 


a(t a V3, (iv) 
sin( £4 =)- “Wis 2 +e(4) 


where E(//2) is the absolute error with interval length A/2. Since 
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equation (iv) gives: 


£(4)-75-05-20 = --0.019618139, 


2) 2 
and then 
=) = 4.45630633. 
E(h/2) 
In a similar way, we obtain the values 
E(hI2) _ 4263856931 
E(hi4) 
and 
ad, = 4,141353027. 
E(h/8) 


The h?-order of convergence is quite revealing in the above results. 


Example 1.12 The Maclaurin expansion for e* is given by 


2 n=l 


xr 9 - 
ome Eee foe alee eee a"; 
ne a at (n—1)! on! ee Ta 


We shall find m, the number of terms, such that their sum yields the value 
of e* correct to 8 decimal places at x= 1. 


Clearly, the error term (i.e. the remainder term) is (x"/n ye*, so that 
at &=x this gives the maximum absolute error, and hence the maximum 
relative error is x"/n! For an 8 decimal accuracy at x= 1 we must have 

1 1 

—_— 

nt 2 
which gives n=12. Thus, we need to take 12 terms of the exponential 
series in order that its sum is correct to 8 decimal places. 


(10) 


EXERCISES 
1.1. Round-off the following numbers to two decimal places: 
48.21416, 2.3742, 52.275, 2.375, 2.385, 81.255. 


1.2. Round-off the following numbers to four significant figures: 
38.46235, 0.70029, 0.0022218, 19.235101, 2.36425 


1.3. Calculate the value of ./ 102 - ,/101 correct to four significant figures. 
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1.4. 


1.6. 


1.8. 
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If u=3v’ —6v, find the percentage error in w at v= 1, if the error 
in v is 0.05, 


Define the term absolute error. Given that 


a=10.002 0.05 
b = 0.0356 + 0.0002 


¢c=15300+100 
d = 62000 + 500, 


find the maximum value of the absolute error in 
(a) atb+e+d (b) a+5ce-d and (c) c’. 
Obtain the range of values within which the exact value of 


1.265(10.21-7.54) 
47 
lies, if all the numerical quantities are rounded-off. 
What is meant by absolute and relative errors? If 


_ 0.31x+2.73 
~ x+0,35 
where the coefficients are rounded-off, find the absolute and relative 
errors in y when x=0.5+0.1. 
Find the sum of the numbers 105.5, 27.25, 6.56, 0.1568, 0.000256, 


208.6, 0.0235, 0.538 and 0.0571, where each number is correct to 
the digits given. Estimate the absolute error in the sum. 


. Find the product of the numbers 56.54 and 12.4 which are both 


correct to the significant digits given. 


. Find the quotient g=x/y, where x = 4.536 and y = 1.32, both x and 


y being correct to the digits given. Find also the relative error in the 
result. 


- Prove that the relative error of a product of three non-zero numbers 


does not exceed the sum of the relative errors of the given numbers. 


. Find the number of terms of the exponential series such that their 


sum gives the value of e* correct to five decimal places for all 
values of x in the range 0<x<1. 


. The function /(x)=tan!x can be expanded as 


yn! 
2n-—| 


Find such that the series determines tan“'1 correct to eight significant 
digits. 


tan trex 4S eae 
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1.14. 


Derive the series 


log (##)=2 pe ee 
"\1-x 35 


and use it to compute the value of log, (1.2) correct to seven decimal 
places. Determine the number of terms required if the series for 
log, (1 +x) were used instead. 


. Write down the Taylor series expansion of f(x)=cos x at x=2/3 


in terms of f(x) and its derivatives at x=2/4, Compute the 
approximations from the zero-order to the fifth order and also state 
the absolute error in each case. 


. The Maclaurin expansion of sin x is given by 


, x xy x? 
sin ¥ =X =-— +— - — +=, 
a! SS! OT! 
where x is in radians. Use the series to compute the value of sin 25° 
with an accuracy of 0.001. 
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Solution of Algebraic and 
Transcendental Equations 


2.1 INTRODUCTION 


In scientific and engineering studies, a frequently occurring problem is to 
find the roots of equations of the form 


f(x) =0. (2.1) 


If f(x) is a quadratic, cubic or a biquadratic expression, then algebraic 
formulae are available for expressing the roots in terms of the coefficients. 
On the otherhand, when f(x) is a polynomial of higher degree or an expression 
involving transcendental functions, algebraic methods are not available, and 
recourse must be taken to find the roots by approximate methods. 

This chapter is concerned with the description of several numerical 
methods for the solution of equations of the form (2.1), where f(x) is 
algebraic or transcendental or a combination of both. Now, algebraic functions 
of the form 

fy (x) = ax” + yx” + agx? 44a, x4 ay, (2.2) 
are called polynomials and we discuss some special methods for determining 
their roots. A non-algebraic function is called a transcendental function, 
e.g., f(x)=In x -0.7, d(x) =e°5* — 5x, w(x) =sin? x-x*-2, etc. The 
roots of (2.1) may be either real or complex. We discuss methods of finding 
a real root of algebraic or transcendental equations and also methods of 


determining all real and complex roots of polynomials. Solution of systems 
of nonlinear equations will be considered at the end of the chapter. 


20 
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2.2 THE BISECTION METHOD 


This method is based on Theorem |.1 which states that if a function f(x) 
is continuous between @ and 4, and f(a) and f(b) are of opposite signs, 
then there exists at least one root between a and 4. For definiteness, let f(a) 
be negative and f(b) be positive. Then the root lies between a and b and 
let its approximate value be given by x» =(a+5)/2. If f(x,)=0, we conclude 
that x» is a root of the equation f(x)=0. Otherwise, the root lies either 
between x, and Af, or between x, and a depending on whether f(xp) is 
negative or positive. We designate this new Interval as [a,, 4,] whose length 
is |-al/2, As before, this is bisected at x, and the new interval will be 
exactly half the length of the previous one. We repeat this process until the 
latest interval (which contains the root) is as small as desired, say ¢, It is 
clear that the interval width is reduced by a factor of one-half at each step 
and at the end of the mth step, the new interval will be [a,, 4,] of length 
lb — ai/2". We then have 


dee 
of 
which gives on simplification 
ny Obello alle) (2.3) 
log, 2 


Inequality (2.3) gives the number of iterations required to achieve an accuracy 
é. For example, if |4-a]=1 and ¢=0.001, then it can be seen that 


n210 (2.4) 
The method ts shown graphically in Fig. 2.1. 


[b, #(b)) 


trl --<-<<<<<=<== 


[a, f(a)] 
Figure 2.1 Graphical representation of the bisection method. 


It should be noted that this method always succeeds. If there are more 
roots than one in the interval, bisection method finds one of the roots. It 
can be easily programmed using the following computational steps: 
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1, Choose two real numbers a and 6 such that f(a) f(6)<0. 

2. Set x, =(a+6)/2. 

3. (a) If f(a) f(x,)<0, the root lies in the interval (a, x,). Then, set 
b=x, and go to step 2 above. 


(b) If f(a) f(x,)>0, the root lies in the interval (x,, 5). Then, set 
a=x, and go to step 2. 

(c) If f(a) f(x,}=0, it means that x, is a root of the equation 
J(x)=0 and the computation may be terminated. 


In practical problems, the roots may not be exact so that condition (c} 


above is never satisfied. In such a case, we need to adopt a criterion for 
deciding when to terminate the computations. 


A convenient criterion is to compute the percentage error £, defined by 
x, =X, 


x; 


&, = x 100%, (2.5) 


where x) is the new value of x,. The computations can be terminated when e, 
becomes less than a prescribed tolerance, say ¢,. In addition, the maximum 
number of iterations may also be specified in advance. 

Example 2.1 Find a real root of the equation f(x) =x° -x-1=0. 


Since f(1) is negative and f(2) positive, a root lies between | and 2 
and therefore we take x, =3/2. Then 


2? 3 («15 


-='_=~-" which is positive. 
F(X) gage. 
Hence the root lies between | and 1.5 and we obtain 
141.5 
= = 1.25 
a] 5 


We find f(x,)=-19/64, which is negative. We therefore conclude that the 
root lies between 1.25 and 1.5. If follows that 


ee 1.25+41.5 
: 2 
The procedure is repeated and the successive approximations are 


xy =1.3125, xy = 1.34375, xs = 1.328125, ete. 


=1.375 


Example 2.2 Find a real root of the equation xi -2x-5=0. 
Let f(x)=x' -2x-—5. Then 
f(2)=-1 and /(3)=16. 
Hence a root lies between 2 and 3 and we take 
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ar amd 


Since f(x) =5.6250, we choose [2, 2.5] as the new interval. Then 
2+2.5 


x= =2.25 and f(x,)=1.890625 


Proceeding in this way, the following table is obtained. 


n a b x f(x) 
1 2 3 25 §.6250 
2 2 2.5 225 1.8906 
a 2 2.25 2125 O.3457 
4 2 2.125 2.0625 0.3513 
5 2.0625 2.125 2.09375 —0,0089 
6 2.09375 2.125 2.10938 0.1668 
7 2.09375 2.10938 2.10156 0.07856 
8 2.09375 2.10156 2.09766 0.03471 
9 2.08375 2.09766 2.09570 0.01286 
10 2.09375 2.09570 2.09473 0.00195 
fh 2.09375 2.09473 2.09424 =0,0035 
12 2.08424 2.09473 


At a= 12, it is seen that the difference between two successive iterates is 
0.0005, which is less than 0.001. Thus this result agrees with condition 
given in (2.4). 


Example 2.3 Find a positive root of the equation xe” =1, which lies 
between 0 and 1. 

Let f(x)=xe* -1. Since f(0)=-1 and f(1)=1.718, it follows that a 
root lies between 0 and 1. Thus, xy =0.5. Since / (0.5) is negative, it follows 
that the root lies between 0.5 and |. Hence the new root is 0.75, Le., 
x, = 0.75. Using the values of xq and x,, we calculate ¢,: 


x) — % 
*y 


* 100 = 33.33%. 


&| = 
Again, we find that /(0.75) is positive and hence the root lies between 0.5 
and 0.75, ie. x; = 0.625, Now, the error is 


_ | 0.625 -0.75 
2 0.625 


Proceeding in this way, the following table is constructed where only the 
sign of the function value is indicated, The prescribed tolerance is 0.05%. 


}+100 = 20%, 
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Sign of 
lteration a b x, F(x;) E, (%) 
1 a] 1 0.5 negative _ 
2 05 1 0.75 positive 33.33 
3 0.5 0.75 0.625 positive 20.00 
4 0.5 0.625 0.5625 negative 11.11 
5 0.5625 0.625 0.59375 positive 5.263 
6 0.5625 0.59375 0.5781 positive 2.707 
fi 0.5625 0.5781 0.5703 positive 1.368 
8 0.5625 0.5703 0.5664 negative 0.688 
9 0.5664 0.5703 0.5684 positive 0.352 
10 0.5664 0.5684 0.5674 positive 0.176 
11 0.5664 0.5674 0.5669 negative 0.088 
12 0.5669 0.5674 0.5671 negative 0.035 


Thus, after 12 iterations, the error, ¢,, finally satisfies the prescribed tolerance, 
viz., 0.05%. Hence the required root is 0.567 and it is easily seen that this 
value is correct to three decimal places. 


2.3 THE METHOD OF FALSE POSITION 


This is the oldest method for finding the real root of a nonlinear equation 
#(x)=0 and closely resembles the bisection method. In this method, also 
known as regula falsi or the method of chords, we choose two points a and 
b such that f(a) and f(4) are of opposite signs. Hence, a root must lie in 
between these points. Now, the equation of the chord joining the two points 
[a, f(a)} and [6, f(6)] is given by 


y— Hla) _ f(6)— fla) (2.6) 


x-a h=a 


The method consists in replacing the part of the curve between the points 
[a, f(a)] and [5, f(6)) by means of the chord joining these points, and 
taking the point of intersection of the chord with the x-axis as an approximation 
to the root. The point of intersection in the present case is obtained by 
putting y=0 in (2.6). Thus, we obtain 
_,- £@) gn -FO-H(a) a 

OF) Fa) F0)= F(a)” mn 


which is the first approximation to the root of f(x)=0. If now f(x,) and 
J (a) are of opposite signs, then the root lies between @ and x,, and we 
replace & by x, in (2.7), and obtain the next approximation. Otherwise, we 
replace a by x, and generate the next approximation. The procedure is 
repeated till the root is obtained to the desired accuracy. Figure 2.2 gives 
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a graphical representation of the method. The error criterion (2.5) can be 
used in this case also. 


Y 


Ala, Fa)] 


Figure 2.2 Method of false position. 
Example 2.4 Find a real root of the equation : 
F(x)= = 2x-5=0. 


We find f(2)=—-1] and f(3)=16. Hence a=2, b=3, and a root lies 
between 2 and 3. Equation (2.7) gives 


x, = LEO =3ED _ 35 _ 5058823529. 
i6-(-) 7 


Now, f(x,) = -0.390799917 and hence the root lies between 2.058823529 and 
3.0. Using formula (2.7), we obtain 


_ 2.038823529(16) — 3(—0.390 799917) 
16.3907999 | 7 


Since f(x) =—0.147204057, it follows that the root lies between 2.08126366 
and 3.0. Hence, we have 


3 2.08 126366 (16) — 3(—0.147204057) 
16.147204057 
Proceeding in this way, we obtain successively: 
Xq = 2.092739575, xs = 2,09388371, 
x, = 2.094305452, xy = 2.094460846,... 


The correct value is 2.0945... so that x; 1s correct to five significant 
figures. 


Xy = 2.08126366. 


X3 = 2.08963921 1. 


Example 2.5 Given that the equation x** = 69 has a root between 5 and 8. 
Use the method of regula-falsi to determine it. 


Let f(x)=x*? -69. We find 
f(5)=-34.50675846 and (8) =28.00586026. 
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Hence 
_ 5(28.00586026) — 8(—34.50675846) 


“1 38.00586026 + 34.50675846 


Now, f(x) =—4.275625415 and therefore, the root lies between 6.655990062 
and 8.0. We obtain 


¥ = 6.83400179, x; = 6.850669653. 


The correct root is 6,8523651..., so that x, is correct to three significant 
figures. 


= 6.655990062, 


2.4 THE ITERATION METHOD 


We have so far discussed root-finding methods, which require the interval 
in which the root lies. We now describe methods which require one or more 
starting values of x. These values need not necessarily bracket the root. The 
first is the iteration method, which requires one starting value of x. 

To describe this method for finding the roots of the equation 


f(x) =90, (2.1) 
we rewrite this equation in the form 
x = (x). (2.8) 
There are many ways of doing this. For example, the equation 
x tx? -1=0 
can be expressed as either of the forms: 
x=(texy 2, x=(1-)?, x=(1-x7)"3,..., 


Let xy be an approximate value of the desired root ¢. Substituting it for x 
on the right side of (2.8), we obtain the first approximation 


x, = (xp) 
The successive approximations are then given by 
=P), = OOD) Xp = PO). 
A number of questions now arise: 
(i) Does the sequence of approximations xp, x),...,x,, always converge 
to some number &? 
(ii) If it does, will € be a root of the equation x = ¢(x)? 


(iii) How should we choose ¢ in order that the sequence xp, x),..., x, 
converges to the root? 


The answer to the first question is negative. As an example, we consider 
the equation 


x=10* +1, 


Presented By: http://www.ebooksuit.com 


SECTION 2.4: The Iteration Method 27 


If we take xy =0, x, =2, xy =101, 2x, =10'"' 41, etc., and as n increases, x, 
increases without limit. Hence, the sequence x», ¥), ¥:, ..., x, does not 
always converge and, in Theorem 2.1 below, we state the conditions which 
are sufficient for the convergence of the sequence. 


The second question is easy to answer, for consider the equation 
Xpet =O(Xq)s (2.9) 


which gives the relation between the approximations at the nth and (n+ 1)th 
stages. As n increases, the left side tends to the root &, and if @ is continuous 
the right side tends to @(¢). Hence, in the limit, we have = 6(¢) which 
shows that € is a root of the equation x= (x). 

The answer to the third question is contained in the following theorem: 


Theorem 2.7) Let x=& be a root of f(x)=0 and let / be an interval 
containing the point x=. Let @(x) and @’(x) be continuous in /, where 
@(x) is defined by the equation x =@(x) which is equivalent to f(x)=0. 
Then if | é’(x)| <1 for all x in J, the sequence of approximations xp, xy, X2, 
...» X, defined by (2.9) converges to the root &, provided that the initial 
approximation xg is chosen in J. 


Proof Since € is a root of the equation x= ¢(x), we have 
f= $(¢) (2.10) 
From (2.9), 
x = G(X) (2.11) 
Subtraction gives 
§ — x = 0(5)- (xp) 


By using the mean value theorem (see Theorem 1.3), the right-hand side 
can be written as (¢ -x9)@'(é9), xp <4) <¢. Hence we obtain 


$-% =(S-%o)8 (So), XH <50 <5 (2.12) 


Similarly we obtain 


S—-* =($-H P(E) 9 <a <e (2.13) 
§—¥3 =(§-%2)6'(E3). << 82<¢ (2.14) 
6 ~ Xn = ($-x,)¢ (Cy dy Xn Sn <5 (2.15) 
If we let 
i@(E)|sk<l, for alli (2.16) 
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then Eqs. (2.12)}{2.15) give 
16 —x%| S/o —29], 1G XQ] eS Hyper es 


which show that each successive approximation remains in / provided that 
the initial approximation is chosen in /. Now, multiplying Eqs. (2.12) to (2.15) 
and simplifying, we obtain 


& —Xnay = (6 —%9) PO (En) O (61).-. P(E), (2.17) 
Since | ¢'(é,)|<*, the above equation becomes 


{F-x, [se |£-x9| (2.18) 


As noo, the right-hand side of (2.18) tends to zero, and it follows that 
the sequence of approximations x), x,,..., converges to the root ¢ if k< 1. 
The method can be represented graphically as follows. By sketching the line 
y=x and the curve y=@(x) and considering the way in which the 
approximations x; are obtained, a geometrical significance of the method is 
obtained and this is shown in Figs, 2.3-2.6. 


y= x 


¥ y=¢@(x) 


OE xy Ry xy 


0 Ny Xp & x, 


Figure 2.4 | @'(x)|<1 but ¢’(x)<0. The process is convergent but the 
approximations oscillate about the exact value. 
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y=9(x) 


O ¢é My My Ky 


Figure 2.5 ¢'(x)>1; the process is divergent. 
¥ 


yux 


Figure 2.6 |¢@°(x)|>1; the process is divergent. 


The root so obtained is unigue. To prove this, let &, and &, be two roots of 
Eg. (2.8), ie. let & =¢(é,) and € =¢(é,). Then we obtain 


|S) — S31 =le(d)- PCE Hl OE - eal, 7 € (6.42) 


which further simplifies to 


| & — €2((l-|¢" (7)\) = 0. (2.19) 


Since | @'(7)|<1, it follows that  =&,, and hence the root is unique. 
Again, 


<[x-9()=1-91@) 


which is positive, since @'(x)<1 in the interval /. This shows that the root 
obtained by this method is a simple root. 
To estimate the error of the approximate root obtained, we have 


IS Xn] =] PG) - PO Sk 1S — Xp ol 
=kl| g—x, +X, —X,-4 | 


Sk|o~x,|+4 |X, ~ Xp 
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which gives 


Al 


fx), Hp] x -%|- (2.20) 


In general, the speed of the iteration depends on the value of &; the 
smaller the value of &, the faster would be the convergence. If ¢ is the 
specified accuracy, i.e. if 


[§—x,] 5 é, 
then formula (2.20) gives 


l-k 
Iq ~ Hil S 6, (2.21) 


which can be used to find the difference between two successive iterates 
necessary to achieve a specified accuracy. The following examples illustrate 
the application of this method. 


Example 2.6 Find areal root of the equation x° +x” -1=0 on the interval 
[0, 1] with an accuracy of 10+ 


To find this root, we rewrite the given equation in the form 


y= res o 
Thus 
Ox)=—er,  g(xy=-t—_! 
Jx+l 2 (x+1y"" 
and 


ax |¢ ()=s4e =k 0.17678 < 0.2. 


[oul 2/8 
Using (2.21) we then obtain 
pos jg OOS 
JC. 0.2 


Hence when the absolute value of the difference does not exceed 0.0004, 
the required accuracy will be achieved and then the iteration can be terminated. 
Starting with x) =0.75, we obtain the following table: 


= 0.0004, 


fn Xn Xn ee 
0 0.75 1.2226756 O.7559289 
1 0, 7559289 1.3251146 0.7546517 
2 0.746517 1.3246326 0.7549263 
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At this stage, we find that 
| Xue] 7X, | = 0.7549263 — 0.7546517 = 0.0002746, 


which is less than 0.0004. The iteration is therefore terminated and the root 
to the required accuracy is 0.7549. 


Example 2.7 Find the root of the equation 2x =cos x+3 correct to three 
decimal places. 


We rewrite the equation in the form 


x= (cos x+3) (i) 
so that 
#() =+ (60s +3), 
and 
t=] <1. 


Hence the iteration method can be applied to the eq. (i) and we start with 
Xg =#/2. The successive iterates are 


x =1.5, xy = 1.535, xy = 1518, 
xy = 1.526, Xs =| .522, x, = 1.524, 
xy =1.523, xg = 1.524. 


Hence we take the solution as 1.524 correct to three decimal places. 
Example 2.8 Use the method of iteration to find a positive root, between 
0 and 1, of the equation xe* =1. 
Writing the equation in the form 
Ea a @) 
We find that é(x)=e * andso @'(x)=-e™*. 
Hence | @'(x)| <1 for x <1, which assures that the iterative process defined 


by the equation x,,,=@(x,) will be convergent. 
Starting with xy» =1, we find that the successive iterates are given by 


x, =e = 0.3678794, x = 0.6922006, 
x; = 0.5004735, x4 = 0.6062435, 
xs = 0.5453957, xg = 0.5796123, 
x, = 0.5601154, xg = 0.571143], 
xq = 0.5648793, X49 = 0.5684287, 
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x1 = 0.5664147, X19 = 0.567556, 
X43 = 0.5669089, X14 = 0.5672762, 
x15 = 0.5670679, xg = 0.567186, 
x7 = 0.567119, xg = 0.567157, 
Xi9 = 0.5671354, Xo = 0.5671477. 


Acceleration of convergence: Aitken's A*-process 
From the relation 


|S — Xn l=]6(S)-P@ GSES — xl, k<l 


it is clear that the iteration method is linearly convergent. This slow rate of 
convergence can be accelerated by using Aitken’s method, which is described 
below. 


Let x;_),X;.Xj;4,; be three successive approximations to the desired root 
x= of the equation x=@(x). From Section 2.4, we know that 


6 -% =k - x4) 5 — X43 = *(E - x) 
Dividing, we obtain 


5 - sa 5 Xi] 
a ee ta 
which gives on simplification 


2 
E=Xy __ i (2.22) 


If we now define Ax, and A*x, by the relations 
Ax,=xj,1-%, and Ax, =A(Ax,), 
then 
A? x, = A(A%)-4) 

= A(x; - 4) 

= Ax, -— Ax, 

= Xin) — Hj — CX) — %4) 

= Xjg) — 2%; + Xj. 
Hence (2.22) can be written in the simpler form 
(ax)" 
hh ae 


=m 2.23) 


which explains the term A*- process. 
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In any numerical application, the values of the following underlined 
quantities must be obtained. 


Xj-] 
Ax, 

Xx; Ars. 
Ax; 

X14] 


Example 2.9 We consider again Example 2.7, viz., the equation 


1 
==(3+ 
x 2! cos x) 


As before, 
ay =1.5 
‘ 0.035 
xy = 1.535 -0.052 
—0.017 
xy =1.518 
Hence we obtain from Eq. (2.23) 
(-0.017)" 
= 1.518 —- ——_—_—_. = 1.524, 
a ase a 


which corresponds to six normal iterations. 


2.5 NEWTON-RAPHSON METHOD 


This method is generally used to improve the result obtained by one of the 
previous methods. Let xy be an approximate root of f(x)=0 and let 
x) =X) +A be the correct root so that f(x,)=0. Expanding f(x, +h) by 
Taylor's series, we obtain 


a 
f(%q) + Af G9) + = Fg) + = 0. 
Neglecting the second- and higher-order derivatives, we have 
faq) + Af Gq) = 0, 


which gives 
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A better approximation than x, is therefore given by x,, where 


Successive approximations are given by x3, %3,....X,4,;, where 
F(X) 
a a . (2.24) 
n+l A f (x,) 
which is the Newfon—Raphson formula. 
If we compare Eq. (2.24) with the relation 
*n+l] =¢(x,) 
of the iterative method [see Eq. (2.9)] we obtain 
f(x) 
fx) 


g(x)=x- 
which gives 


roy — LOS") (2.25) 
g(x) - LOL) 
TOR 


To examine the convergence we assume that f(x), f‘(x) and f"(x) are 
continuous and bounded on any interval containing the root x=€ of the 
equation f(x)=0. If € is a simple root, then =f'(x) #0. Further since f‘(x) 
is continuous, | /'(x)|=¢ for some ¢>0 in a suitable neighbourhood of é. 
Within this neighbourhood we can select an interval such that | f(x) f"(x)| <<? 
and this is possible since f(€)=0 and since f(x) is continuously twice 
differentiable. Hence, in this interval we have 


| #'(x)| <1. (2.26) 


Therefore by Theorem 2.1, the Newton—Raphson formula (2.24) converges, 
provided that the initial approximation x») is chosen sufficiently close to €. 
When € is a multiple root, the Newton-Raphson method still converges but 
slowly. Convergence can, however, be made faster by modifying formula 
(2.24). This will be discussed later. 

To obtain the rate of convergence of the method, we note that f(¢)=0 
so that Taylor’s expansion gives 


Fp) +E — 9) F Oy) +5 Ein) fog) 4° = 0, 
from which we obtain 


_fGn) _ = dee 2f"Gy) 237 
FG) (¢ nt 5 a) Fe.) (2.27) 
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From (2.24) and (2.27), we have 


ei, ae 
Tint é = 5 (x, é) T'&) 
Setting 
Ey = Xy —§ 
Equation (2.28) gives 
| re 
én 588 


(2.28) 


(2.29) 


(2.30) 


so that the Newton-Raphson process has a second-order or quadratic 


CONnVerpence. 


Geometrically, the method consists in replacing the part of the curve 
between the point [x5, /(x,)] and the x-axis by means of the tangent to the 
curve at the point, and is described graphically in Fig. 2.7. It can be used 
for solving both algebraic and transcendental equations and it can also be 


used when the roots are complex, 


¥ 


Figure 2.7 Newton—Raphson method. 


Example 2.10 Use the Newton—Raphson method to find a root of the 


equation x° -2x-5=0. 


Here f(x) =x° —2x—5 and f'(x)=3x* —2. Hence Eq. (2.24) gives: 


<7 — 2x, —5 


x =x," 
n+l A 
3x2 —2 


(i) 


Choosing x» =2, we obtain f(xy)=—-I and f'(xy)=10. Putting mn =0 in (i), 


we obtain 
% -2-(-L)=21 
10 
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Now, 
F (m4) =(2.1" -2(2.1) -5 = 0.061, 
and 
f(x) =3(2.1" -2 =11.23. 
Hence 


x, = 21-2! 2.994568. 
11.23 


This example demonstrates that Newton—Raphson method converges more 
rapidly than the methods described in the previous sections, since this 
requires fewer iterations to obtain a specified accuracy. But since two 
function evaluations are required for each iteration, Newton—Raphson method 
requires more computing time. 
Example 2.1] Find a root of the equation xsin x+cos x=0, 
We have 
J (x)=xsinx+cosx and /'(x)=xcos x, 
The iteration formula is therefore 
X, Sin xX, + COS x, 
x) =X, oo, 
n+] n Xp, COS Xp 


With xy) =, the successive iterates are given below 


n Xn F(Xq) Kno 

0 3.1416 -1.0 2.8293 
1 2.8233 0.0662 2.7986 
2 2.7986 0.0006 2.7984 
3 2.7084 0.0 2.7084 


Example 2.12 Find a real root of the equation x =e", using the Newton— 
Raphson method. 


We write the equation in the form 
f(x)=xe* -1=0 (i) 
Let x) =1. Then 


e-t if. t 
=}—2£=! _1/),1).9.6830307 
cee? (+4) 


Now 
f(s) =0.3553424, and F(a) = 3.337012, 
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so that 
xy = 0,6839397 — 29953424 _ 9 sa74sqs, 
3.337012 


Proceeding in this way, we obtain 
x, =0.5672297 and x, =0.5671433. 


Generalized Newton's method 


If € is a root of f(x)=0 with multiplicity p, then the iteration formula 
corresponding to (2,24) is taken as 


Spot =%p — Pe, (2.31) 
F'(%,) 
which means that (1/p)/'(x,) is the slope of the straight line passing through 
(X,,}',) and intersecting the x-axis at the point (x,,), 0). 

Equation (2.31) is called the generalized Newton's formula and reduces 
to (2.24) for p = 1. Since ¢ isarootof f(x)=0 with multiplicity p, it follows 
that € is also aroot of f"(x)=0 with multiplicity (p—1), of 7#"(x)=0 with 
multiplicity (p—2), and so on. Hence the expressions 


f(x) 
, %-(-7Lw 
Fay 7 P- FG) 


must have the same value if there is a root with multiplicity p, provided that 
the initial approximation x, is chosen sufficiently close to the root. 


Example 2,13 Find a double root of the equation f(x) =. -x* —x+1=0. 
Choosing x, =0.8, we have 
f'@)=3e -2x-1, and f(x) =6x-2. 
With xp = 0.8, we obtain 


and 


The closeness of these values indicates that there is a double root near to 
unity. For the next approximation, we choose x, =1.01 and obtain 


x, -2£22 -1.01- 0.0099 = 1.0001, 
Pm) 
and 


_F@) _ 1 .91~0,0099 = 1.0001 
xj f"(4) , . ; 
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We conclude therefore that there is a double root at x=1.0001 which is 
sufficiently close to the actual root unity. 


On the other hand, if we apply Newton-Raphson method with x, = 0.8, 
we obtain 


x, =0.8+0,1060.91, and x, =0.91+0.046 = 0.96, 


[t is clear that the generalized Newton's method converges more rapidly 
than the Newton—Raphson procedure. 


2.6 RAMANUJAN’S METHOD 
Srinivasa Ramanujan (1887-1920) described an iterative method* which 
can be used to determine the smallest root of the equation 
f(x) =, (2.1) 
where (x) is of the form 
f(xj=1 ~(aqx+ayx" +x + ax" eee), (2.32) 
For smaller values of x, we can write 
[l—(ayxtagx? tae tayxt+. JP! =b +bxt ax? +. (33) 
Expanding the left-hand side by binomial theorem, we obtain 
1+ (ayx + ayx? + ayx° +o) + (ayx + ax? + ayx? +°--)? += B + byx + byx? fees 
(2.34) 


Comparing the coefficients of like powers of x on both sides of (2.34), we 
pet 


& =1, 
by =a, = a5, 


by = ay +a, = aby + anh, (2.35) 


b, = ab,_) + 2,8, 2 +--+ +a, m= 2,3,... 


Without any proof, Ramanujan states that the successive convergents, 
viz., 6,/b,,;, approach a root of the equation f(x)=0, where f(x) is given 
by (2.32). The following examples illustrate the application of this method. 


Example 2.14 Find the smallest root of the equation 
f(x)=x - 6x7 +11x-6=0. (i) 


*See Berndt [1985], p.41. 
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We have 
2 a0 
fy uerseoe| = by thyx+byx? ++ (ii) 
Here 
at) By 
nl gt a, =-l, “8? ay =a, =: =0 
Hence, 
& =]; 
1] 
ale Bale 
121 85 
by = aby + anh, 36 36° 
575 
b = =—; 
4 = aby + aby + a3, = 
3661 
=a,b = 
bs = aby + aby + ayby + agd, 1296: 
22631 
bg = abs + aaby + yb; + ayby + Osh =i 
Therefore, 


85 
= us = 0.8869565 

bg 

by 3450 

— =—— =(.9425654 
be 3661 

os = ae = 0.9706155 
be 3233 


The smallest root of the given equation is unity and it can be seen that the 
successive convergents approach this root. 
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Example 2.15 Find a root of the equation xe* =1. 
Let xe* =] (i) 


Expanding e* in ascending powers of x and simplifying, we can rewrite 
eq. (i) as: 

3 4 se 

lexevr stele dy (ii) 


6 24. 
which is of the form of the right side of Eg. (2,32). Here 


l l l 
a, = 1, a =, ay =a a a as yu 


b =I, 

by = a, =1; 

by = a,b, + ayb =14+1=2; 
= Gyby + Gyby + ab = 2+1+ 


re l 
bs = ab, + dab; + a,b, + ay ta ee 
7 


11 261. 


+ dab, + +ash =— I nS ce! 
be = abs + ayby + a,b, + aby + ash, +— rh + ri 4 


Therefore, 


tl 
o 
‘Lara 


Hl 
= 
LA 
=] 
— 
pele 


i il 

| ee ee 
us 

Es 

g 

a 


LS ae oe 
Koad 
| 


It can be seen that Newton’s method (see Example 2.12) gives the value 
0.5671433 to this root. 


Example 2.16 Find a root of the equation sin x =1—x. 
Using the expansion of sin x, the given equation may be written as 
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120 5040 
Here 
a, = 2, a, =0, ay =-— ay =0, 
i dee. ee 
“5 = 150° 6 7” 5040" 


We then obtain 


b=1 
b, =a, =2; 
by = aby + ag = 4; 


1 4/7 
by = aybs + ayby + ash =8- = =—: 


a 


Oy = caybyy + apy + cays + gh = 


3601 
be = aby + aby + ayb, + aghy + acl = 50 


Therefore, 


= 0.5106382 
27 


RS ad Pd ad Rc 
ll 
Ie 


47 
=—— =().5] 
7 0.5108695 
p= S61 705109681 


The root, correct to four decimal places, is 0.5110. 
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Example 2.17 Using Ramanujan’s method, find a real root of the equation: 
2 4 
Pe See, se, 
(29? GN? (4 


4 
PS (Pree sete: ee Sed (i 
(2n? 3? (4p? 


To apply Ramanujan’s method, we write 


Sad | 
h-[s-2+ a. rae | = by + byx+byx? ++ (il) 


(2) @n? (4) 
Here 
; Be 
a an? 3 BY 
eae oe 1 : 1 
4p? (51)? o (6n? 
Hence, we obtain 
b=1 
b, =a, =1, 
wen 
by = aby +h =1- 


ee f 31,1 


a ae ee 


a (21)? ay 4 4 36 


21 As 


_ 21 
~ 576 


bd 
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It follows 


oe eH 1, 4210.00, 
b, 4 19 19 
ea SL YT 
be 36 2! 


where the last result is correct to three significant figures. 
This example demonstrates that Ramanujan’s method is preferable when 
the given function consists of an infinite series. 


2.7 THE SECANT METHOD 


We have seen that the Newton—Raphson method requires the evaluation of 
derivatives of the function and this is not always possible, particularly in the 
case of functions arising in practical problems. In the secant method, the 
derivative at x, is approximated by the formula 


F(x) - fi) 


xX; — XW] 


FQ) 
which can be written as 


fiz fi-fin (2.36) 


where jf, = f(x;). Hence, the Newton—Raphson formula becomes 
Si -%) _ Mev ~ fin | (2.37) 


Xi4l LE —>OoO=e_—X—X—X—s—Ss— 


Ki-Ft hi-fi 


It should be noted that this formula requires two initial approximations to 
the root. 


Example 2.18 Find-a real root of the equation x° — 2x -5=0 using secant 
method. 


Let the two initial approximations be given by 
x.,=2 and xp =3. 
We have 
f(x_y)=f,=8-9=-1, and f(x) = fp = 27-11 =16. 
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Putting ¢=0 in (2.37), we obtain 


x= 2016) — 31) _ 35 _ 9 958823529. 
17 17 


Also, 
f(x) =f, =0.390799923. 
Putting i=1 in (2.37), we obtain 


of — *fg _ 3(-0.590799923) ~ 2.058823529(16) _ » 09126366 


aaa EE A ~16.390799923 


Again 

f (2) = fo =-0.147204057. 
Setting i= 2 in (2.37), and simplifying, we get x; = 2.094824145, which is 
correct to three significant figures. 


2.8 MULLER’S METHOD 


In this method, f(x) is approximated by a second-degree curve in the 
vicinity of a root. The roots of the quadratic are then assumed to be the 
approximations to the roots of the equation f(x}=0. The method is iterative, 
converges almost quadratically, and can be used to obtain complex roots. 

Let x,_3,%;;.x, be three distinct approximations to a root of f(x) =0. 
Let yj), ¥j-1> ¥, be the corresponding values of y = f(x). Now, any second- 
degree curve passing through the point (x,, y,) may be written as 


p(x)= A(x-x,)" + B(x-3x,)+ y;. (2.38) 


Since the curve also passes through (x;_3, );2) and (x,_), ¥;,), We 
obtain 


Vina = A(H-2 — HP + BR — Hy) +s (2.39) 

and 
Yo = ACH — 8)" + BC — 81) + Y- (2.40) 

Solving (2.39) and (2.40), we get 
is ¥i-1 ~ Yi ni ¥i-2 — Fi (2.41) 
(4 —Xj9) O44 3%) Oe — 4) 04-2 - 4-1) 

and 

B= 2h1—Hi _ A(x, -x,) (2.42) 
Aj — ¥j 


With these values of 4 and &, the quadratic Eq. (2.38) gives the next approxi- 
mation x;,;: 
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Z # 
-Bty B ~4 Ay (2.43) 


2A 


A direct solution from (2.43) leads to inaccurate results and therefore we 
take the equivalent form : 


Apel = A; + 


2yi 
Xjyy = — 7 co 
B+) B? -4Ay, 


in (2.44), the sign in the denominator should be chosen so that the denominator 
will be largest in magnitude. With this choice, Eq. (2.44) then gives the next 
approximation to the root. The following example illustrates the application 
of this method. 


Example 2.29 Find the root of the equation p(x)=x?- 2x—5=0, which lies 
between 2 and 3. 


Let x,.4=1,%).;=2 and x,=3. Then y,3=-6, y;.,;=—1 and y, =16. Hence 
gee. =#0 =17-11=6, 
W-1 (-2)(-) 
and 
-17 


eer a Be le 


The quadratic equation is given by 
6(x-3) +23(x-3)+16=0, 
which gives the next approximation: 
2(16) 


Xj =3— ’ 
23 ‘a (237 - 4(6) (16) 


where the positive sign is chosen since 8 is positive. 
Hence 


= 2.086799548. 


xt 


ee ae 
234/145 
The error in the above result is given by 
| 2.086799548 — 3 

2.086799548 


which simplifies to 43.76%. Since this is quite large, we proceed to the next 
iteration with 


x 100%, 


x9=1 x )=2, x, =2.086799548. 
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The corresponding values of y are 

¥j-2 = —6, ye =o y, = —0.086145588. 
Using these values in Eqs. (2.41) and (2.42), we obtain 

A=5.086799558 and 8=10.96986336. 
The next approximation x,,, is obtained as 
2(0,.086145588) 
22.01933047 

The error in this result is 0.373553519%. 


X41 = 2086799548 + = 2.09462409, 


2.9 GRAEFFE’S ROOT-SQUARING METHOD 


This is another method usually recommended for the numerical solution of 
polynomial equations. It, however, suffers from the disadvantage of mone 
a numerically complicated procedure. 

Let P.(x) be a polynomial of degree nm. Graeffe’s method consists in 
transforming P,(x) into another polynomial, say Q,(z), of the same degree 
but whose roots are the squares of the roots of the original polynomial. The 
process is repeated so that the roots of the new polynomial are distributed 
more spaciously. This is possible provided that the roots of the original 
polynomial are all real and distinct. The roots are finally computed directly 
from the coefficients. We consider, for example, 


P,(x) = (x41) (x-2) (x +3) (2.45) 
Then 
P,(—x) = (-x + 1) (-x - 2) (-x +3) (2.46) 
= (=I) (x-1) (x +2) (x-3). (2.47) 
Hence, 
Py(x) Py(—x) = (-1)° (x? -1) (x? - 4) (x* -9). (2.48) 
Thus, we obtain 
Q,(z) =(z-1)(z-4)(z-9), (2.49) 
where 
z= x’. 


It is seen that the roots of (2.49) are the squares of the roots of the 
original polynomial /3(x). 


Again, 
OQ, (-z) = (-z -1)(-z- 4) (-z - 9) (2.51) 
=(-l) (z+1)(z+4)(z4+9). (2.52) 
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Hence 
QO, (2) O(-z) = (-1P (2? - 1) (z* - 16) (2 -81) (2.53) 
The next new polynomial is therefore given by 
S3(u)=(u —1) (uv = 16) (u 81), (2.54a) 
where 
uaz? (2.54b) 


Suppose that the above procedure is repeated m times (i.e. the squaring 
is done m times successively); then the roots are estimated from the formula: 


Lim 


(i=1, 2, ...,7), (2.55) 


where the a,’s are the coefficients of the new polynomial (obtained after 
squaring m times) and » is the degree of the original polynomial. 

it is clear from the above discussion that this method gives approximations 
to the magnitudes of the roots. To determine the signs of the roots, we 
substitute each root in the original polynomial and find the result. If the 
result is very nearly zero, then the root is positive, otherwise it is negative. 
The method of application ts tllustrated below. 


Example 2.20 Find the real roots of the equation x° -6x* +11x-6=0. 
Let 


P,(x) = x" — 6x" +1 1x-6. (i) 
Then 
Py(-x) = (-xy’ -6(-xy° +11(-x)-6 
= (-1)' (x? +6x7 +1 1x +6). (ii) 
Hence 
P.(x) P,(-x) =(-1P (x° = 14x* + 49x? - 36). (iii) 
Let 
O, (z)= 27 - 1427 + 492-36, (iv) 


where z =x", Therefore, roots of P,(x)=0 are given by 


| 36 49 Vv 
= = 0.857, — =1,871, 14 =3.741. 
0.85 (v) 


Again 
QO, (-z) = (1) (2 +142? + 492 + 36). (vi) 
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Hence, 
O;(z) Q,(-z) = (-1P (2° - 9824 +13932? - 1296). (vii) 
It follows that 
S;(u) =u? —98u7 +1393u — 1296, (viii) 


where u=z* =x", From the roots of the new polynomial §,(u)=0, we 
obtain the approximation to the roots of A(x)=0 as 


4 va 
1296 1393 
| =0,9822, 1399) 1.942, 4 
] ( og (98) =3.147. (ix) 


The convergence to the exact roots, 1, 2 and 3 is quite clear. 


2.10 LIN-BAIRSTOW’S METHOD 


A method which is often useful in finding quadratic factors of polynomials 
is Lin-Bairstow’s method and this is briefly described below: 
Let the polynomial be given by 
f(®) = Ayx? + Agx + Ax + Ay =0. (2.56) 


Let x7+Rx+S be a quadratic factor of f(x) and also let an approximate 
factor be x* +rs+.s. Usually, first approximations of r and s can be obtained 
from the last-three terms of the given polynomial. Thus, 


4 ‘aii aoe (2.57) 


f(x) = (x? +945) (Box +B) +Cx+D 


= Bax? +(Byr + B,)x* +(C + Byr+sB,)x+(Bs+D), (2.58) 


where the constants #,, #5, C and D have to be determined. Equating the 
coefficients of the like powers of x in Eqs. (2.56) and (2.58), we obtain 


a (2.59) 


From (2.59), it is clear that the coefficients B of the factored polynomial 
and also the coefficients C and D are functions of r ands. Since x* + Rx +S 
is a factor of the given polynomial, it follows that 
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C(R, §)=0 and DR, Sj=0 (2.60) 
Letting 
Rer+Ar and S=s+As (2.61) 
Eqs. (2.60) can be expanded by Taylor's series 
C(RS)=C(r,8) +r S + ds S =o 
(2.62) 
ap oD 
D(R,5)= D7, s)+ Ar — + As — =D, 
( } ir, s)+ a + as 


where the derivatives are to be computed at r and s. 

Equations (2.62) can then be solved for Ar and As. Use of these values 
in (2.61) will give the next approximation to RX and S, respectively. The 
process can be repeated until successive values of & and 8 show no significant 
change. The following example illustrates the application of this method. 


Example 2.21) Find the quadratic factor of the polynomial given by f(x) = 
3 2 
x —2x* +x-2. 


We have 4; =1, 4; =—2, 4) =1 and Ay =—2. It is easily seen that r=~1/2 
and s=1, Equations (2.59) give 


B=! 

By =-2-r; 

C=l-r(-2-r)-s=1+2r+r’-s, 

D=-2-s(-2—-r)=-2 + 25 +865. 
Also, 


! 3 
[CO N12, 0 Se orl 


] l 
Dir, sy] e<-24+2--=--; 
[Ptr N12, a 


(=) =2+2r=1; 
oF } (12,1) 


(s) 
(e) 
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Equations (2.62) give: 


igacapes Jd eo ae: 
4 2 2 
Hence 
20 10 
Thus 
i018 5 
2 20 20 
and 
pth? oa 
10 #10 


It follows that the quadratic factor is x° +0.15x+0.9. Thus, for the second 
approximation, the approximate quadratic factor is x° + 0.15x+0.9 so that 
r=0.15 and s=0,9. Now, 


C =1+2.15(0.15) -0.9 = 0.4225, 
D= -2 + 2.15(0.9) = 0.065, 
OC/ér = 2+ 2(0.15) = 2.30, 
Clas = —1, 
éDiér = 0.9, 
ODlas = 24+ r=2.15. 
Hence, Eg. (2.62) give 
2.34r—-As=—0.4225 and 0.9Ar+2.15As =-0.065 
Solving the above equations, we obtain 
Ar =—0.1665312, As =0.0394785, 
The second approximations are therefore given by 
R =0.15 -— 0.16653 12 = -0.0165312 
5 =0.9 + 0,0394783 = 0,9394783 


Thus the second approximation to the quadratic factor of f(x) is 
x*—0.0165312x + 0.9394783, and it can be seen that R and S are approaching 
their actual values 0 and 1, respectively. 
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2.11 THE QUOTIENT-DIFFERENCE METHOD 


This is a general method to obtain the approximate roots of polynomial 
equations and is originally due to Rutishauser [1954]. We adopt here the 
notation and methodology described in Henrici [1958]. The procedure is 
quite general and is illustrated here with a cubic polynomial. Let the given 
cubic equation be 


F(x) = ax? + ax" + @x+a, =0 (2.63) 


and let x,,x, and x, be its roots such that 0 <|x,|<|x){<|x,|. Then f(x) can 
be expressed in the following way: 


r=] *~*r pal “rx, Xp 
on 
=)) &%, (2.64) 
ia) 
where 
: P (2.65) 
= r b 
= ae =? i+1° 
Fa Xp 


The method derives its name to the gwotients g'") and the differences A‘! 
defined by the relations 


g(t (2.66) 
i-1 
and 
Alo ~ g(t = gi (2.67) 
We have already seen that 
lim Zi) (2.68) 
io oF 


tends to the smallest root of the equation f(x)=0 (see, Section 2.6). The 
quotient-difference method is therefore an extension of Ramanujan’s method 
arid determines all the real roots of a polynomial equation. Using the definitions 
(2.66) and (2.67), starting values of g{” and A\” can be found and these 
are used to generate a table of quotients and differences from the general 
formulae: 


| aes 
At aft sae) gy (2.69) 
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and 
A +9 alt agi” (2.70) 


40) = AM) =0, for all i, n being the degree of the polynomial. These formulae 
can easily be established using the definitions and their proofs are left as 
exercises to the reader (see, Henrici [1974]). A typical quotient-difference, 
table is given in Table 2.1. 


Table 2.1 A Typical Quotient-difference Table 


Ay mn Ay qe A q Ag 


q? ay a2 

a a®) att ate) 
qf? gf) qf? 

42) Ag 4 a? 
qh q)! q3” 

aS) a@) al a) 


If the first-two rows are known, the succeeding rows can be generated using 
the formulae (2.69) and (2.70) alternately. To determine the differences A“'*), 
formula (2.69) is used whereas formula (2.70) determines the quotients 
qi"). As the building-up of Table 2.5 proceeds, the quantities g{", g{ 
aa. tend to the reciprocals of the roots of the cubic equation (72. é3), 
However, instead of (2.63), if we consider the transformed equation 


ax" + ax" + ajx+ ay =0 (2.71) 


and proceed as above, we obtain, directly, the roots of Eq. (2.63). The 
following example illustrates the method of procedure : 


Example 2.22 Find the real roots of the equation x° -6x° +1lx-6=0. 
Let 


fi) =x -6x7 +1 1x-6=0. (i) 
To obtain the roots directly, we consider the transformed equation 
—6x° +11x* -6x+1=0 (ii) 
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l — j 
ae jx = Oy +ayx+ ctx" +ayx? foes 
—6x° +1ix" -6x+1 jp 


Hence, 
(-6x° + 11x? —6x4 1) (ay + ax +anx" + 03x" fees) (in) 
Comparing the coefficients of like powers of x on both sides of (ii), we 
obtain 
a =1, a, —6a@, =0, a, —6a, + lla =0, a, — 6a, +1 la, — 6a) =0, etc 
The above equations give 
a, = 6, a@, =25, a, =90,... 


Hence 
q(? = ay/ay = 6.00000, 
qi” = ayia, = 25/6 = 4.16667, 
qi?) = aslay = 90/25= 3.60000 
Again, 
AM = 9) _ gl -_1.83333, 
and 


AY) = 9/9) — 9!) = 0.56667. 
Now, in the rhombus (see Table 2.1), 
a 


2 
qi q3 


ai) 
all the quantities except qi) are known, Taking r=1, and j=1, formula 
(2.69) gives 

(2) (2) 

Ay ’g 
1 an et 
93) =——ay— = 1.28789. 

A} 

Next, we need to determine the value of A”), For this, we consider the rhombus 


Li 
qs” 


af Aw) 
a? 
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in which all the quantities, except A‘) are known. Using formula (2.70), 
we obtain 


a) =q!) +All -q 
= 1,28789 -1.83333, since g$) =0 
= —().54544. 
Also, 
ACY 50. 


Thus, all the quantities in the first-two rows are known and the succeeding 
rows can be built-up, using the formulae (2.69) and (2.70), alternately. The 
numerical results are shown in the following table: 


An KN Ay q2 Aa 3 Ag 
6.00000 0 0 

0 —1,63333 —0.54544 0 
4.16667 1.28788 0.64544 

0 —0. 56667 —).23100 0 
3.60000 1.62956 0.77644 

0 —0.25556 0.11047 0 
o.34444 1.76865 0.88691 

0 0.13515 —0.05540 a] 
a.20829 1.8484 0.94231 

0 —O.07 784 0.02824 0 
3.13145 1.898 0.9706 


It is evident that g,, g, and g3 are gradually converging to the roots 3, 2 
and 1, respectively. 


2.12 SOLUTION TO SYSTEMS OF NONLINEAR EQUATIONS 


In this section, we consider the solution to simultaneous nonlinear equations 
by two methods: (i) the method of iteration and (ii) Newton—Raphson method. 
In both the cases, the first approximations are usually obtained from a 
graph. For simplicity, we consider the case of two equations in two unknowns. 


2.12.1 The Method of Iteration 
Let the equations be given by 
f(x, y)= 9, g(x, y)=0, (2.72) 
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whose real roots are required within a specified accuracy. As in the method 
of iteration for a single equation (see Section 2.4), we assume that the 
equations in (2.72) may be written in the form 


x= F(x, y), y=G(x, v), (2.73) 


where the functions F and G satisfy the conditions: 


oF 


OG 


ox 


OG 


oy 


oF 
ax 


+ <1 and rr <] (2.74) 


in the neighbourhood of the root. 

Let (xp, 4) be the initial approximation to a root (¢, 7) of the system 
(2.72). We then construct the successive approximations according to the 
following formulae: 


x, = F (x9, ¥o); ¥ =G (Xp, Yo) 
x4 = F(x.34) vy =G(x, 4) 


x; = F(x,, ¥5), ¥y = G(X5, V5) (2.75) 


Aneel = Fixns Vad ¥ nel =G(Xqs Yn) 


For faster convergence, recently computed values of x; may be used in the 
evaluation of y, in (2.75). If the iteration process (2.75) converges, then we 
obtain 


é=F(e.m) and 9 =G(5,7) (2.76) 


in the limit. Thus, € and 7) are the roots of the system (2.73) and hence, 
also, of the system (2.72). Conditions (2.74) are sufficient for the convergence 
of the iteration (2.75). We state the following theorem: 


Theorem 2.2 Let x=€ and y= 7 be one pair of roots of the system 
(2.73) in the closed neighbourhood &. If the functions F and G and their 
first partial derivatives are continuous in R, 


OG 


ay 


for all (x, y) in A, and the initial approximation (x», y9) is chosen in X&, then 
the sequence of approximations given by (2.75) converges to the roots 
x=€ and y= of the system (2.73). 

The proof of this theorem is similar to that of Theorem 2.1 and is 
omitted here. The method can obviously be generalized to any number of 
equations, 


oF <1 and 
ax 


<1 (2.77) 


= 
+|— 
Gy 
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Example 2.23 Find a real root of the equations: 
x=0.2x7+0.8,  y=0.3xy? +0.7 
We have 
F(x, y)=0.2x7 +08, G(x, y)=0.3x)* +0.7 
Then 


=0.4x, —<=0 


=03y*, 2 _ 0.6xy. 
ay 


It is easy to see that x =1 and y=1 are the roots of the system. Choosing 
Xo = jo = V2, we find that 


=02<1 


a (%, oo a 


and 


= 0.225 <1 


el 
(30, Yo) 


(Xp, a 


Thus, conditions (2.77) are satisfied. Hence, 


x =F (%q. 9) = "= +0.8=0.85 


4 =G (xo. 0) =“ 0.7=0.74 
For the second approximation, we obtain 
xy = F(x, »;)= 0.2% (0.85)" +0.8 = 0.9445 
y2 = G(x, 4) =0.3 « (0.85) « (0.74)? +0.7 = 0.81. 


Convergence to the root (1, 1) is obvious. In computing the y values in the 
above, we could have used the recently computed values of x for a faster 
convergence. For example, 


y, =G(%, ¥g) =0.3x ; x (0.85) = 0.764, 


which is a better approximation than the previously computed one. 
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2.12.2 Newton-Raphson Method 


Let (xp, Yo) be an initial approximation to the root of the system (2.72). If 
(%) +h, vo +k) is the root of the system, then we must have 


f(xg +A, yg +k) =, 2(x +h, yo +k) =0 (2.78) 


Assuming that f and g are sufficiently differentiable, we expand (2.78) by 
Taylor’s series (see Theorem 1.6, Section 1.2) to obtain 


to htrkZ4...20 


z os (2.79) 
g g 
+hp——+h#——+:02=0, 
ee Gry Vy 
where 
ff | Ff _ 
xz. Jo = F(X. Yo), ete. 


Neglecting the second- and higher-order terms, we obtain the following 
system of linear equations: 


—=-f, 
ai % (2.80) 
og , & 
h——=+k—=- 
ay Oy 
If the Jacobian 
xy F 
od 
Jf, g)= (2.81) 
& & 
ce cy 
does not vanish, then the linear Eqs. (2.80) possess a unique solution given 
by 
| + < x 
h= and 4=—!_| ® (2.82) 
J(f, 2) a og J(f.2)| og e 
oy Ar 
The new approximations are then given by 
x, =Xq +A, y=Notk (2.83) 
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The process is to be repeated till we obtain the roots to the desired 
accuracy. If the iteration converges, it does so quadratically. Theorem 2.3 
below (stated without proof) gives the conditions which are sufficient for 
convergence: 


Theorem 2.3 Let (xo, ¥o) be an approximation to a root (€, 7) of the system 
(2.72) in the closed neighbourhood # containing (¢,7). If (a) 4 2 and all 
their first and second derivatives are continuous and bounded in R, and 
(b) J/(f,z)#0 in A, then the sequence of approximations given by 


. ag d ‘i f 

Kip FX) — and y= 7,- a of (2.84) 
JF, 2) 

(Ff, 2)\) > ; ; JCF, 2) 


converges to the root (¢,17) of the system (2. i 


Example 2.24 Find areal root of the equations x?-y?=3 and x? +y?=13. 
For starting the solution, we take y= x as our ne approximation. This 
gives 


Xo =Yy = (6.5 = 2.54951, 


and therefore 
fo=-3 and gy, =0, 
where 
f=x'—y?-3 and gex?+y?-13. 
Further, 


= 2x, = 5.09902, 


Ff og 

ax arg 

Ff __3 = =5,09902 & 4 = 5,09902. 
yp Yo By Yo 


Hence 


fF Ff 

dy = Avy |_| 5.09902 — -5.09902 

ag ag | (5.09902 5.09902 | 

Cy = Hy 

and therefore the convergence criterion is satisfied. We then have 
(5.09902) + k(-5.09902)=3 and (5.09902) + & (5.09902) = 0. 


These equations give 
h=0.29417 and k=-0.29417, 
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Hence the first approximation to the root is given by 
Xy = Xp + A= 2.54951 + 0.29417 = 2.84368 
¥ = Yo tk = 2.54951 — 0.29417 = 2.25534. 
For the second approximation, we have 
A, =F (es ¥) = -0.000042573 
2, = g(%, ¥) = 0.173074458. 


of af 
= 2x, = 5.68736, —=-2y) =-4.51068, 
A 1 ay, ¥| 


| 
Og cg 
—=2x, =5.6873 —=2y = 4.51068. 
Ox) * i yy ‘ 


Clearly, the condition of convergence is satisfied and we have the simultaneous 
equations 
A(5.68736) + k(—4.51068) = 0.000042573 
and 
A(3.68736) + &(4.51068) = —0.17307. 


Solving the above equations, we obtain 
h=-0.01521 and f= -0.01919. 
the second approximation is therefore given by 


x = 2.84368 -— 0.01521 = 2.82847, 
and 
yy = 2.25334 - 0.01919 = 2.23615. 
The above values may be compared with the true values, which are given by 
x= /8=2.82843 and y=./5 =2.23607. 


EXERCISES 


Obtain a root, correct to three decimal places, for each of the following 
equations using the bisection method (Problems |—10): 


2A. x 4x7 4047250 2.2. x°-4x-9=0 
2.3. x? -x-4=0 2.4. x -18=0 

2.5. x) -x*-1=0 2.6. x +x7-1=0 
2.7. x3-3x-5=0 2.8. x? -x-1=0 
2.9. x? -5x4+3=0 2.10. x +x-1=0. 
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Use the method of false position to obtain a root, correct to three decimal 
places, of each of the following equations (Problems 11-15): 


zZ.11. 


2.13. 


2.15. 


4x74x4+7=0 2.12. x -x-4=0 


xx" -1=0 2.14. x? -x-1=0 


4x-1=0, 


Use the iteration method to find, correct to four significant figures, a real 
root of each of the following equations (Problems 16—25): 


2.16. 
2.18. 
2.20. 
2.22. 


2.24, 
2.26. 


cos x=3x—-1 2.17. x=M(x4+1)P 

x=(5-x)" 2.19. sinx =10(x-1) 

e* =10x 2.21. xsinx=1.0 

sin? x =x? - 1 2.23. 2 =cot x 

l+x7=x° 2.25. 5x'-20x+3=0. 

Compute a root of the equation e* =x" to an accuracy of 10°, 


using the iterative method. 


Use Newton-Raphson method to obtain a root, correct to three decimal 
places, of the following equations (Problems 27—36): 


Z.2T, 


2.29. 
2.31. 


2.33. 


2.35. 
2.37. 


2.38. 


2.39. 


z.40. 


xf _ 4 ap 2.28. sinx=1-x 

x? -5x+3=0 2.30. y44.2-80=-0 
437 -1=-0 2.32. 4(x-sin x)=1 
x-cosx=0 234. sinx=(1/2)x 
x+logx=2 2.36. xe?* =(1/2) sin x 
Establish the formula 


] N 
re | 3(. +2] 
2 x; 


and hence compute the value of a2 correct to six decimal places. 
Find the least positive root of the equation tan x =x to an accuracy 
of 0.0001 by Newton—Raphson method. 

Obtain, to four decimal places, the root between | and 2 of the equation 
x -2x? +3x-5=0 by (a) Regula—Falsi, (b) Newton—Raphson 
method. 


Using Ramanujan’s method, obtain the first-eight convergents of 
the equation x+x° =1. 
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2.41. 


2.42. 


2.43, 


2.44, 


2.45. 


2.46. 


2.47. 


2.48. 


2.49. 


200. 
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Using iteration method, find the real root of the equation 
y= x x x* 
ay oF GP 
Solve the same equation using Ramanujan's method and compare 
the results. 
Using iteration method, find the real root of the equation 
ey x x. a 
| *T0 42 “316 


Use the secant method to determine the root, lying between 5 and 
8, of the equation x?? =69. Compare your result with that obtained 
in Example 2.5. 

Determine the real root of the equation xe* =1 using the secant 
method. Compare your result with the true value of x = 0.567143... 


Apply Graeffe’s root-squaring method to determine the approximate 
roots of the equations: 


(a) x ~2x" -x+2=0 (b) x? -7x? +10x-2=0. 
Use Muller’s method to find a root of the equations: 
(a) x -x-1=0 (b) x3 -x7-x-1=0. 


Using Lin—Bairstow’s method, obtain the quadratic factors of the 
following equations: 

(a) x -2x7 +x-2=0 (b) x4 +5x° +3x* -5x-9=0. 
Apply the quotient-difference method to obtain the approximate 
roots of the equations: 


(a) x -2" —~2x4+1=0 (b) Problem 45(b). 
Prove that 
(a) lim A. = i 

ie yy Ay 


x, —a i? 
rig Pre eer fie 
ime (aie) om OP xy 


Ux, (i+) 
rE eae a a 1 et 
ies (x, / x5) * Py ¥y 


in the notation of Section 2.11. 

Develop a subprogram (in any computer language of your choice) 
for the Newton-Raphson method indicating the maximum number 
of iterations and also the tolerance for the percentage error in your 
solution. Test your program on Problem 36. 
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Zee 


2.53. 


2.54. 


2.55. 
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Develop a subprogram for computing roots by Muller’s method. 
Use Problem 46(b) to test your program. 


Use a package of your choice to determine all the roots of the 
polynomial equation 
fx) =(2 41) (4-2) (2 -— 4) (x + 6) = 0. 


Develop a subprogram to implement Bairstow’s method and test it 
on Problem 47(b). 

MATLAB has a subprogram called roots (c) for calculating the roots 
of a polynomial, whose coefficients are stored in the vector (c) and 
to return the roots to a vector (r). Use this program to obtain the 
roots of the equation 


x4 —5x5 +9.25x7 -7.75x4+2.5=0. 
The following equation occurs in rocket dynamics: 


Mig (1 — eta, J=uyt, 


where my is the mass of the rocket at time f=0, v is its upward 
velocity at time ft seconds, v, is the relative velocity at which the 
fuel is ejected, wu; is the fuel consumption rate and g is the acceleration 
due to gravity (= 9.8 m/sec’). Determine ¢ (to within 1% of the true 
value) when v=1500 m/sec, mg = 200,000 kg, v, = 2500 m/sec and 
u, = 3000 kg/sec, using any subprogram of your choice. 


Solve the following systems of nonlinear equations by any suitable method 
(Problems 56—58): 


2.56. 
2.57. 
2.58. 


wany a4, x+y? =16. 
x +yell, yr+x=7. 
x =3xy-7, y= 2(x4+]). 
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CHAPLER 


Interpolation 


3.1 INTRODUCTION 


The statement 


y= f(x), % SXSX, 

means: corresponding to every value of x in the range x» <¢x<x,, there 
exists one or more values of y. Assuming that f(x) is single-valued and 
continuous and that it is known explicitly, then the values of f(x) corresponding 
to certain given values of x, say xp, X|, .-..%, can easily be computed and 
tabulated. The central problem of numerical analysis is the converse one: 
Given the set of tabular values (xp, ¥o), (X),.¥ 1). (425 ¥2)s ---+ Otp Vp) Satisfying 
the relation y= f(x) where the explicit nature of f(x) is not known, it is 
required to find a simpler function, say @(x), such that f(x) and @(x) agree 
at the set of tabulated points. Such a process is called interpolation. If ¢(x) 
is a polynomial, then the process is called polynomial interpolation and 
g(x) is called the interpolating polynomial. Similarly, different types of 
interpolation arise depending on whether @(x) is a finite trigonometric series, 
series of Bessel functions, etc. In this chapter, we shall be concerned with 
polynomial interpolation only. As a justification for the approximation of an 
unknown function by means of a polynomial, we state here, without proof, a 
famous theorem due to Weierstrass (1885): if,f(x) is continuous in xp =x = x,, 
then given any ¢>0, there exists a polynomial P (x) such that 


| f(x)- P(x)|<e, for all x in (xp, x,). 
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This means that it is possible to find a polynomial P (x) whose graph remains 
within the region bounded by y= f(x)-—« and y= f(x)+<e forall x between 
x) and x,, however small « may be. 

When approximating a given function f(x) by means of polynomial (x), 
one may be tempted to ask: (i) How should the closeness of the approximation 
be measured? and (ii) What is the criterion to decide the best polynomial 
approximation to the function? Answers to these questions, important though 
they are for the practical problem of interpolation, are outside the scope of 
this book and will not be attempted here. We will, however, derive in the 
next section a formula for finding the error associated with the approximation 
of a tabulated function by means of a polynomial. 


3.2 ERRORS IN POLYNOMIAL INTERPOLATION 


Let the function y(x), defined by the (+1) points (x,, y,), /=0, 1, 2,....4, 
be continuous and differentiable (n+1) times, and let y(x) be approximated 
by a polynomial ¢,(x) of degree not exceeding n such that 

O,(x))=y,, %§=0,1,2,...,7 (3.1) 


If we now use ¢,(x) to obtain approximate values of y (x) at some points 
other than those defined by (3.1), what would be the accuracy of this 
approximation? Since the expression y(x)-@¢,,(x) vanishes for x = x9,x),...,%)5 
we put 


Wx) = 9, (x) = Lr, )(%), (3.2) 


where 


Ane \(X) = (X — Xp) (X -%)--- XH) (3.3) 


and £ is to be determined such that Eq. (3.2) holds for any intermediate 
value of x, say x=x', x) <x' <x,. Clearly, 


par ad alo hl (3.4) 
yy (X) 
We construct a function F(x) such that 
F(x) = v(x) —@,(%) — Laty,)(), (3.5) 


where £ is given by Eq. (3.4) above, 
It is clear that 
F(x) = F(x) =-)= F(x,) = F(x") = 0, 


that is, F (x) vanishes (m + 2) times in the interval x, <x <x,,; consequently, by 
the repeated application of Rolle’s theorem (see Theorem 1.3, Section |.2), F'(x) 
must vanish (m+ 1)times, F"(x) must vanish va times, etc., in the 
interval x) x x,. In particular, Fox) must vanish once in the interval, 
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Let this point be given by x= €, xy <¢ <x,. On differentiating (3.5) (n+ 1) 
times with respect to x and putting x=, we obtain 


O= py) (e)- Ln + 1)! 


so that 
{n+l} 
per) (3.6) 
(n+l)! 
Comparison of (3.4) and (3.6) yields the results 
ine) 
my r _¥ (¢) r 
y(x') sonal lar PS Tp Tn (©). 
Dropping the prime on x’, we obtain 
y(x)-¢, (x) = Tne (X) mst) (ey Xy < & or. (3.7) 


(n+1)! 


which is the required expression for the error. Since »(x) is, generally, 
unknown and hence we do not have any information concerning 
yx), formula (3.7) is almost useless in practical computations. On the 
other hand, it is extremely useful in theoretical work in different branches 
of numerical analysis. In particular, we will use it to determine errors in 
Newton's interpolating formulae which will be discussed in Section 3.6. 


3.3 FINITE DIFFERENCES 


Assume that we have a table of values (x,, y,), /=0, 1, 2,....1 of any function 
y= f(x), the values of x being equally spaced, i.e. x, =xy +ih, i=0, 1, 2,..., m1. 
Suppose that we are required to recover the values of f(x) for some 
intermediate values of x, or to obtain the derivative of f(x) for some x tn 
the range x9 Sx<x,. The methods for the solution to these problems are 
based on the concept of the ‘differences’ of a function which we now proceed 
to define. 


3.3.1 Forward Differences 


If Yos dis S242 ¥_ denote a set of values of y, then ¥) — 0992 — Mees —¥n-t 
are called the differences of y. Denoting these differences by Ayy, Ay,,.... AV, 
respectively, we have 


Ayg=¥— Yor AME Fa Mees AY = Pn Fs 
where A is called the forward difference operator and Ayo, Ay,..., are 
called first forward differences. The differences of the first forward differences 
are called second forward differences and are denoted by A? Yo» A? Mipsis 
Similarly, one can define third forward differences, fourth forward differences, 
etc. Thus, 
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A? yg = Ay, - Avy = 2-1 -O1- Yo) 
= 2-2) + Yo» 
A? yy = A? y, - A? yp = ys - 29 +. — (2-2) + Yo) 
=¥y—3¥2 +3y - Yo 
At yg = A*y, — A’ yy = ¥4— 393 +392 — Wi - (39-32 +3 - Yo) 
= V4 — 43 + 6 yy — 4, + Yy- 


It is therefore clear that any higher-order difference can easily be expressed 
in terms of the ordinates, since the coefficients occurring on the right side 
are the binomial coefficients. 


Table 3.1 shows how the forward differences of all orders can be formed: 


Table 3.1 Forward Difference Table 


x y¥ ‘A a® a” at Am Ae 
og Yo 
AYp 
a A A* yo 
Ay Ady, 0 
xg ¥2 AP y, At yo 
AY Avy, Ayo 
Kg 7) Ayo Asy, AS yo 
Ays Ar yo Any 
x Ya A*y, A“ yo 
A¥4 A* yg 
x5 ¥5 A®y, 
AYs 
xg ye 


3.3.2 Backward Differences 


The differences ¥) —¥o.¥o —¥isees Fn ~¥n-) are called first backward 
differences if they are denoted by Vy,, Vyy,.... Vy, respectively, so that 
Vy =o Veo = 2 — os VP = ¥en Fn) Where V is called the 
backward difference operator. In a similar way, one can define backward 
differences of higher orders. Thus we obtain 


V7 yy = Voy - Wy = 2-1 -O1- Yo} = 2 - 2, +0» 
Vy = V7 y5 - V7 yp = 5 -— 392 +34 — Yo» ete 


With the same values of x and y as in Table 3.1, a backward difference table 
can be formed: 
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Table 3.2 Backward Difference Table 


x y v v v v4 ve ve 
ag ¥o 

x4 ¥4 v4 

Ko V9 V¥2 V2 

xq V3 V3 v4y, VW ys5 

xg Ya Vy¥4 ve va Wv4 V"y4 

x5 Ys VWs v°y5 Ws vvs V5 

XG ¥6 We Ve Wye Ve V'¥—5 Ve 


3.3.3 Central Differences 
The central difference operator & is defined by the relations 


Mw Yo=O8¥y2, Yao M=OVa Qe Mn ~¥n-t =O¥n- 


Similarly, higher-order central differences can be defined. With the values of 
x and y as in the preceding two tables, a central difference table can be 
formed: 


Table 3.3 Central Difference Table 


x y & Be s a s gf 
XQ Yo 
dyye 
x Val b*y4 
dyai2 5" y 3/0 
Xa Yo yo 54yo 
O¥s/2 5° Y5i0 BP yspo 
Xs Ys 5° ya 54 ya 8° yg 
OY 72 5 ya/0 By a79 
X4 Ya dy, 54y4 
oyga S*yg/2 
Xs ¥5 é* ¥5 
Ova 
x6 Ye 


It is clear from the three tables that in a definite numerical case, the 
same numbers occur in the same positions whether we use forward, backward 
or central differences. Thus we obtain 


Avy = Vy, =8yy,95 A yy = Vy. =O yar yee 
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3.3.4 Symbolic Relations and Separation of Symbols 


Difference formulae can easily be established by symbolic methods, using 
the shift operator £ and the averaging or the mean operator #, in addition 
to the operators, A, V and J already defined. 

The averaging operator y is defined by the equation: 


l 
BY, = 5 Orati2* Yr-12 ). 
The shift operator £ is defined by the equation: 


Fy, = Vr+ds 
which shows that the effect of E is to shift the functional value y, to the 
next higher value y,,,. A second equation with £ gives 


E*y, = E(Ey,) = Evp41 = Yar) 


and in general, 


E" Vr = Veen 
It is now easy to derive a relationship between A and £, for we have 


Ayo =) — ¥o = Evo — ¥o =(E-1) ¥ 
and hence 
A=E-| or E=1+A. (3.8a)* 


We can now express any higher-order forward difference in terms of 
the given function values. For example, 


Ay =(E-I)’ yy = (E? -3E” + 3E-l) yp = ¥3 — 392 +3 - Yo. 
From the definitions, the following relations can easily be established: 
V=l-E", 
6 = fU2_ pi 
(3.86) 
p= (1/2) (BN? + EM?) ge? =14: (0/4) 87 
A=VE=6E"", 
As an example, we prove the relation u* = 1+(1/4)5*. We have, by definition, 
I 
Yr = 5 rar + ¥p—112) 
1 = 
= 5 (Ey, +E y,) 


*The student should note that Eq. (3.8a) does not mean that the operators E 
and A have any existence as separate entities; it merely implies that the effect 
of the operator Eon yp is the same as that of the operator (] +A) on yp. 
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=; (E24 ami )y,- 


Hence 
at (gl2y g-¥2) 
2 
and 
ws u (EN? 4 gl/2y2 
4 
! -| 
= ri (E+E ° +2) 
7 ; (eM? — e-N2? 4 4) 
= 7 (8? +4). 


We therefore have 


] 22 
= /l+—d". 
vey 4 


Finally, we define the operator D such that 


iy: 
Dy (x)= a ). 


To relate D to £, we start with the Taylor's series 


F hr i hr iF 
PETA a Het TS Y O)+>¥ (x) +-+ 
This can be written in the symbolic form 
2 33 
boc [1oAoe aoe +o | p(x), 


Since the series in the brackets is the expansion of e"°, we obtain the 
interesting result 


Euce", (3.8c) 


Using the relation (3.8), a number of useful identities can be derived. This 
relation is used to separate the effect of £ into that of the powers of A and 
this method of separation ts called the method of separation of symbols. The 
following examples demonstrate the use of this method. 


Example 3.1 Using the method of separation of symbols, show that 


Awe n(n —1) 
i-f 


ft 
=U, —Mu,_) + Wy_7 +" +(-l) u,_,,. 
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To prove this result, we start with the right-hand side. Thus, 
natal) 
z 


u, —mu,y_) + Mig $07 +(-1)"u,_, 


=u, -nEu, + Eu, pi (HI Ew, 


EF pe (Tf e*| Uy 


= AN wins 


which is the left-hand side. 
Example 3.2 | Show that 


: x 
at + xAily +5 Arua +t + uyx+ ail 
Now, 


7 va? 
x 2 x A 
cut +34 wrefee fsx 5 om 


which is the required result. 
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3.4 DETECTION OF ERRORS BY USE OF DIFFERENCE TABLES 


Difference tables can be used to check errors in tabular values. Suppose 
that there is an error of +1 unit in a certain tabular value. As higher 
differences are formed, the error spreads out fanwise, and is at the same 
time, considerably magnified, as shown in Table 3.4. 


Table 3.4 Detection of Errors using Difference Table 


y A a® a? at a® 

0 
0 0 

0 0 
0 0 0 

0 0 1 
0 0 1 

0 1 +4 
0 1 4 

1 3 10 
1 -2 6 

= 3 ~10 

0 1 4 

0 - 5 
0 0 1 

0 0 1 
9 0 0 

0 0 
0 0 

0 
0 


This table shows the following characteristics: 
(i) The effect of the error increases with the order of the differences. 


(ii) The errors in any one column are the binomial coefficients with 
alternating signs. 


(iii) The algebraic sum of the errors in any difference column is zero, and 


(iv) The maximum error occurs opposite the function value containing 
the error. These facts can be used to detect errors by difference 
tables. We illustrate this by means of an example. 
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Example 3.3 Consider the following difference table: 


x ¥ A Ae a® as 

1 3010 
Aid 

z a424 =36 
ava =a 

3 S602 75 +178 
S03 +139 

4 4105 + 64 =27T'1 
KT a4 

5 4472 - 68 +181 
aug +49 

6 S771 =19 — 46 
280 +3 

ri 5051 -16 
264. 

§ 5315 


The term —271 in the fourth difference column has fluctuations of 449 and 
452 on either side of it. Comparison with Table 3.4 suggests that there is 
an error of -45 in the entry for x = 4. The correct value of y is therefore 
4105 + 45 = 4150, which shows that the last-two digits have been transposed, 
a very common form of error. The reader is advised to form a new difference 
table with this correction, and to check that the third differences are now 
practically constant. 

If an error is present in a given data, the differences of some order will 
become alternating in sign, Hence, higher-order differences should be formed 
till the error is revealed as in the above example. If there are errors in 
several tabular values, then it is not easy to detect the errors by differencing. 


3.9 DIFFERENCES OF A POLYNOMIAL 


Let )(x) be a polynomial of the mth degree so that 
¥(x) = gx" +x! + ax 2 4+ 4,. 

Then we obtain 

yx +h) p(x) = aay [(x + AY" x") + a [e+ hy x + 
= dy (nh) x"! +ajx™* +--+ al, 
where a), @>,...,@, are the new coefficients. 
The above equation can be written as 

Ay(x)=ay(nh)x"! +ajx"? +--+ ay, 


which shows that the first difference of a polynomial of the mth degree is 
a polynomial of degree ( — 1). Similarly, the second difference will be a 
polynomial of degree (n — 2), and the coefficient of x"~* will be agn(n — 1h? 
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Thus the ath difference is agn!h", which is a constant. Hence, the (7 +1)th, 
and higher differences of a polynomial of mth degree will be zero, Conversely, 
if the nth differences of a tabulated function are constant and the (7 + 1th, 
(n+ 2)th,..., differences all vanish, then the tabulated function represents a 
polynomial of degree ». It should be noted that these results hold good only 
if the values of x are equally spaced, The converse is important in numerical 
analysis since it enables us to approximate a function-by a polynomial if its 
differences of some order become nearly constant. 


3.6 NEWTON'S FORMULAE FOR INTERPOLATION 


Given the set of (n+ 1) values, ViZ., (xg, ¥o), (Xp. My) CX q 0 Vp dye een (Xp Veg) 
of x and y, it is required to find y,(x), a polynomial of the nth degree such 
that y and y,(x) agree at the tabulated points. Let the values of x be equidistant, 
Le. let 


X,=xy+ih, i=0, 1, 2,.... 7. 
Since y,(x) is a polynomial of the mth degree, it may be written as 
Yq (X) = dy + aj (% ~— Xq) + a2 (X — XQ) (X-¥) 
+ a3(x = xp) (x- x) (6-2 ) 4 (3.9) 
+ a, (x — Xp) (x — x) (X — xy)... — 2,4). 


Imposing now the condition that » and y,(x) should agree at the set of 
tabulated points, we obtain 


A~Yo Ao. g 2890. g 24M... g _ Ao. 
m-% A h2 3! “hn! 


ay = Hos 2 ™ 


Setting x=x)+ ph and substituting for ap,a),...,a,, Eq. (3.9) gives 


_| (p-l)(p-2) .3 
yal8) = Yi + py + PP: dary, +2 P = Pat A ye ++ 
+ PPD pnd (Pant any, (3.10) 
Hy 


which is Newfon’s forward difference interpolation formula and is useful for 
interpolation near the beginning of a set of tabular values. 

To find the error committed in replacing the function y(x) by means of 
the polynomial y,(x), we use formula (3.7) to obtain 


(X=) 7) En) ore (zy, 


(n+1)! My <G<x,. (3.11) 


y(x)— ¥, (4) = 


As remarked earlier we do not have any information concerning yy), and 
therefore formula (3.11) is useless in practice. Nevertheless, if yay) 
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does not vary too rapidly in the interval, a useful estimate of the derivative 
can be obtained in the following way. Expanding »(x +A) by Taylor's series 
(see Theorem 1.4), we obtain 


F hr’ re 
yO +h)= ye) + fy'a)+ Sy (x)+e- 
Neglecting the terms containing A* and higher powers of A, this gives 
l l 
y(x)* i [v(x +h)- y(x)] = 7 Ay(x). 


Writing y'(x) as Dy(x) where D=didx, the differentiation operator, the above 
equation gives the operator relation 
1 


Delta andso D"! 2— aml, 
hh Arel 
We thus obtain 
] 
yD (x) 9 par 8). (3.12) 
Equation (3.11) can therefore be written as 
l 2 a 
y(x)— y, (x)= PPT 2). We) (3.13) 


(n+l)! 
in which form it is suitable for computation. 
Instead of assuming y,(x) as in (3.9) if we choose it in the form 
Yq(X) = ag + aj (% — X,) + ag (x — x) — X41) 
+ 03(%— x, ) Ge — % yg) (8 — yg) to 
+ a, (x —x,,) (4 — %,_))...(— 2). 


and then impose the condition that y and y,(x) should agree at the tabulated 
POINTS X,,X__py-++s Fo. %1,Xg, WE Obtain (after some simplification) 


Ynl2)= Iu + PV + PEED V2, 4.1.4 PPE API D ony, (3.14) 


nr! 
where p =(x—x, )/A. 

This is Newton's backward difference interpolation formula and it uses 
tabular values to the left of y,. This formula is therefore useful for interpolation 
near the end of the tabular values. 

It can be shown that the error in this formula may be written as 


Ha) ~ yyla) = PPO PED HMYOME), GAS) 


where x) <G<x, and x=x, + ph. 


Presented By: http://www.ebooksuit.com 


SECTION 3.6: Newton's Formulae for Interpolation 75 
The following examples illustrate the use of these formulae. 


Example 3.4 Find the cubic polynomial which takes the following values: 
(1) = 24, y(3) =120, (5) = 336, and »(7)= 720. Hence, or otherwise, obtain 
the value of »(8). 


We form the difference table: 
x y A 4 A 


1 24 
96 
3 120 120 
216 48 
5 336 168 
384 
7 720 


Here 4 =2, With xp =1, we have x=1+2p or p=(x-1)/2. Substituting this 
value of p in Eq. (3.10), we obtain 


(4) (2 ] 
x-l 2 2 2 2 Z 

= oe Kes pS, er | 
y(x) = 24+ 5 (96)+ (120)+ ; (48) 


=x +6x7 +1 1x +6. 


To determine »(8), we observe that p= 7/2. Hence, formula (3.10) gives: 


(7/2)(7/2=1) (7/2)(7/2 - 1) (7/2— 2) 
2 6 


Direct substitution in y(x) also yields the same value. 

Note: This process of finding the value of » for some value of x outside the 
given range is called extrapolation and this example demonstrates the fact 
that if a tabulated function is a polynomial, then both interpolation and 
extrapolation would give exact values. 


¥(8) = 24+ 7(96) + (120) + (48) = 990. 


Example 3.5 Using Newton's forward difference formula, find the sum 
\e =P Da aenrtay, 
We have 
Say =P +P 4 4-4 +(n4ly 


Hence 
A 


ne] 


S, =(n+ly, 
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or 
AS, =(n+l). (i) 
It follows that 
A*S, = AS.) ~AS, =(n+ 2) —(n 41) =3n? +9n+7, 
AMS, =3(n +1)? +9n+7 - (3 +904 7) =6n +12 
AtS,, = 6(n+1)+12-(6n+12) =6. 
Since A‘S, =A°S, =---=0,S, is a fourth-degree polynomial in n. 
Further, 
Sj=], AS,=8, A*S,=19, A395, =18, AS, = 6. 
Formula (3.10) gives 
$, =1+(a-1)(@)+ 2-VE-2) we =Tipy Sw oe) mate =) 118) 


Z (n—1)(n-2)(n-—3)(n = 4) (6) 
24 


at ei 
2 4 


= 
4 
~[ne 
ae ae 


Example 3.6 Values of x (in degrees) and sin x are given in the following 
table: 


x (in degrees) sin x 
15 0.2588190 
20 0.3420201 
25 0.4226183 
30 0.5 
35 0.5735764 
40 0.642 7876 


Determine the value of sin 38°. 
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The difference table is 


x sin x A A* A° a’ as 
15 0,2588190 
0.083204 1 
20 «03420201 ~0,0026029 
0,0805982 ~0,0006136 
25 0.4226183 ~0,0032185 p.0000248 
00773817 ~0,0005888 0,0000041 
3 005 -0,0038053 o.0000289 
0.0735764 ~0,0008599 
35 05735764 ~0,0043852 
0.0692112 
40 —«-0.6427876 


To find sin 38°, we use Newton's backward difference formula with x, =40 


and x =38. This gives 
px kzin 238-40 _ 2 og 
h 35 5 


Hence, using formula (3.14), we obtain 


y(38) = 0,6427876 —0.4 (0,0692112)+ ae 


(—0.0043652) 
(—0.4)(—0.4 + 1)(-0.4 + 2) 


‘ (—0.0003599) 


, 0.4) (0.4 + 10.4 + 2)(—0.4 + 3) 
24 
, COA)(C04 +1)(-0.4 + 2)(-0.4+3)(-0.4 + 4) 
120 


(0.0000289) 


(0.0000041) 


= 0,6427876 — 0.02768448 + 0.00052382 + 0.00003583 
— 0.00000120 


= 0.6156614. 


Example 3.7 Find the missing term in the following table: 


# a 
0 1 
| 3 
Z 9 
z = 
4 $1 


Explain why the result differs from 3° = 27? 
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Since four points are given, the given data can be approximated by a 
third degree polynomial in x. Hence A*y, =0. Substituting A= E£-1 and 
simplifying, we get 

E“yy ~4F? yp + 6E* yo ~4 Fy, + yy =0. 
Since E’ yg = y,, the above equation becomes 
Ya ~ 4y3 +692 ~ 4, + Yq =O. 
Substituting for yp, 4,2 and y, in the above, we obtain 
yy =31. 
The tabulated function is 3* and the exact value of (3) is 27. The error is 


due to the fact that the exponential function 3° is approximated by means 
of a polynomial in x of degree 3. 


Example 3.8 The table below gives the values of tan x for 0.10 s x <0.30: 


x y=tanxs 
0.10 0.1003 
0.15 0.1311 
0.20 0.2027 
0.25 0.2553 
0.30 0.3093 


Find: (a) tan 0.12 (b) tan0.26, (c) tan0.40 and (d) tan 0.50. 
The table of difference is 


x ¥ A Af Ay a‘ 
0.10 0.1003 
0.0508 
0.15 0.1514 0.0008 
0.0516 0.0002 
020 0.2027 O.0010 0.0002 
0.0526 0.0004 
0.25 0.2553 0.0014 
0.0540 
0.30 0.3093 


(a) To find tan (0.12), we have 0.12 = 0.10 + p(0.05), which gives p = 0.4, 
Hence formula (3.10) gives 
0,4(0.4—-1) 


tan (0.12) = 0.1003 + 0.4(0.0508) + (0.0008) 
0.40.4 pes =2) 9002) 
m 0.4(0.4 -1) S — 2)(0.4 -3) (0,0002) 


= 0.1205. 
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(b) To find tan (0.26), we have 0.26=0.30+ p(0.05), which gives 
p=-0.8. Hence formula (3.14) gives 


tan (0.26) = 0,3093 — 0.8(0,0540) + 


SAS 0.0014) 


, 70-8(-0.8 +DCO8+2 enn 
4, 70-8(-0.8 + 1)(-0.8+ 2)(-0.8 + 3) 


0.0002 
24 ( ) 


= 0.2662. 


Proceeding as in the case (1) above, we obtain 
(c) tan (0.40) = 0.4241, and 


(d) tan (0.50) = 0.5543. 


The actual values, correct to four decimal places, of tan (0.12), tan (0.26), 
tan (0.40) and tan (0.50) are respectively 0.1206, 0.2660, 0.4228 and 0.5463. 
Comparison of the computed and actual values shows that in the first-two 
cases (i.¢. of interpolation) the results obtained are fairly accurate whereas 
in the last-two cases (i.e. of extrapolation) the errors are quite considerable. 
The example therefore demonstrates the important result that if a tabulated 
function is other than a polynomial, then extrapolation very far from the 
table limits would be dangerous—although interpolation can be carried out 
very accurately, 


3.7 CENTRAL DIFFERENCE INTERPOLATION FORMULAE 


In the preceding section, we derived and discussed Newton's forward and 
backward interpolation formulae, which are applicable for interpolation near 
the beginning and end respectively, of tabulated values. We shall, in the 
present section, discuss the central difference formulae which are most 
suited for interpolation near the middle of a tabulated set. The central difference 
operator 6 was already introduced in Section 3.3.3. 

The most important central difference formulae are those due to Stirling, 
Bessel and Everett. These will be discussed in Sections 3.7.2, 3.7.3 and 
3.7.4, respectively, Gauss's formulae, introduced in Section 3.7.1 below, 
are of interest from a theoretical stand-point only. 


3.7.1 Gauss’ Central Difference Formulae 
Gauss’ forward formula 


We consider the following difference table in which the central ordinate is 
taken for convenience as y) corresponding to x= xp. 
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The differences used in this formula lie on the line shown in Table 3.5. 
The formula is, therefore, of the form 


Vp = Yo + Gy +GpA*y_1 +G,A7y, +GyA*ygt--, (3.16) 


where G,,G,... have to be determined. The y, on the left side can be 
expressed in terms of y), Ay) and higher-order differences of jp, as follows: 


Table 3.5 Gauss’ Forward Formula 


x y A Ae Ng at a? aa 
x3 ¥-3 
4y_g 
x3 ¥-2 ¥_3 3 
Ay_s ‘“ A" y_3 F 
xy ¥-1 A*y_o A’y¥_3 
Ay_4 : A°y_p P 
xq IST a HN fs ae WN, “8 yo, 
yi 
my ¥4 A? Yo a‘ ¥-4 
Ayy A*yo 
x2 ¥o AM yy 
Aya 
ag ¥3 
Clearly, 


=(1+A)? yp, using relation (3.8a) 


a) Pip-l) 42). + PERI P AD yy 4 


Similarly, the right side of (3.16) can also be expressed in terms of y), Ay 
and higher-order differences. We have 


= Yo + PAY + 


A’y_,=A°E'yo 
=A7(1+ A)! yy 
=A7(1-A+A?-AP +) 
= A? (yg —Ayy + A? yy - AY +>) 


=A" yp -A’ yp +A*yp = AP yp ++: 
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AP y_) = A yg — At yy + Ay — APY +> 
Af V2= ret Yo 
= A‘(1+ A)? yo 
= A* (yy —2Ayp +3A7 Yq - 40° 9 ++) 
= A* Vp - 2A° yp +3A° yp - 4A" yo Sa 
Hence (3.16) gives the identity 
PUP=D 2y, - wer - 2) Ay, 


Yo + PAY + aa 


Pe ip=3} 4, 


qeaee 


= Vo + G Avg + Gy(A7 yp -AP yy + A* yp -A’ yy | 


+ Gy(A> yy -A* yp + A? yo -A° yo fees) 


+ G4(A* yo — 2A” Yo + 3A° yy —4Al YQ te) ee (3.17) 


Equating the coefficients of Ayp,A*¥p.A° yo, etc., on both sides of (3.17), 
we obtain 
Gy; = P, 


G3 _ Plp-l) 


2! 


G, et) pip-) aoe 


3! 


g, -hP*) plip-D(p~2) 
. 4! 


Gauss’ Backward Formula 
This formula uses the differences which lie on the line shown in Table 3.6. 
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Table 3.6 Gauss' Backward Fornula 


x y A a® a’ a‘ a® a® 
x4 ¥-4 
Ay_y EIN ON 
Xo re il Nvey3 ay 5 aby 4” 
4y¥o Ay. AP yo 
| ¥4 


Gauss’ backward formula can therefore be assumed to be of the form 
Vy = Yq + GiAy_y +GpA7y_, + GA? yg +GyA* yg + (3.19) 


where Gj, G3,... have to be determined. Following the same procedure as 
in Gauss’ forward formula, we obtain 


Gi =p, 
_ Plp+)) 
a 2! 
,_ (p+!) p(p-}) 
a. 3! 


gi -P+2(P+D p(p-) 
_~ 4! 


i] 


(3.20) 


Example 3.9 From the following table, find the value of e'-'” using Gauss" 
forward formula: 


= 


x € 
1.00 2.7183 
1.05 2.8577 
1.10 3.0042 
1.15 3.1582 
1.20 3.3201 
1.25 3.4903 
1.30 3.6693 


We have 
1.17 =1.15 + p(0.05), 
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which gives 


0.02 1 


Po05 4 
The difference table is given below, 
x e* A Ae Ae aa 
1.00 2.F 18a 
0.1394 
1.05 2.8577 0.0071 
0.1465 OC 
1.10 4.0042 0.0075 0 
0.1540 0.0004 
1.15 3.1 0.0079 0 
a ee 0.0004. 
1.20 3.9201 0.0083 0.0001 
0.1702 0.0005 
1.25 9.4903 0.0088 
0.1790 
1.30 9.6693 


Using formulae (3.16) and (3.18), we obtain 
ell? = 3.1582-+ (0.1619) + C= 0.0079) 


= 3.1582 + 0.0648 — 0.0009 


= 3.2221. 


3.7.2 Stirling’s Formula 
Taking the mean of Gauss’ forward and backward formulae, we obtain 


Ay. +Ayo , p 2-1) Ay) +A*ys 
wy poe 2 ty A y y 


¥p 2 2 31 2 


iy 2 
(p—l) 
+E aty.s ae (3.21) 


Formula (3.21) is called Stirling's formula. 
3.7.3 Bessel’s Formula 


This is a very useful formula for practical interpolation, and it uses the 
differences as shown in the following table, where the brackets mean that 
the average of the values has to be taken. 


Presented By: http://www.ebooksuit.com 


a4 CHAPTER 3: Interpolation 


Hence, Bessel’s formula can be assumed in the form 
+ A*y +A? 
Yp= OSA + Bidyy + By + Bud 


4 4 
+ By PAA Pah 4 


(3.22) 


Using the method outlined in Section 3.7.1, ie. Gauss’ forward formula, we 
obtain 


I 
ie ta 


_ Plp-D 
ma 2! 


_ Plp-))(p-l/2)) (3.23) 


5 3! : 


B _ (p+) p(p-)(p-) 
ai 4! : 


Hence, Bessel's interpolation formula may be written as 


-}) A*y, +A” -1)(p-U 
Yp= Yo + Payy + PSE— vat Yo, PUP a 2) 3y 


12+) PP-D(p-2) A*y-2 Aya dt (3.24) 
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3.7.4 Everett's Formula 


This is an extensively used interpolation formula and uses only even order 
differences, as shown in the following table: 


% Yo a’y A*y_o A®y_s 


x, v4 A? yo Aty, A®y_» 


Hence the formula has the form 
Vp= Ep yg + EA? y_) + EgA* yg +++ + Foy + yA yg + FA" y_y ++ 3.25) 


The coefficients E,, Fy, E,, 5, Eq, Fy,... can be determined by the same 
method as in the preceding cases, and we obtain 


Ey =1-p=4, Fy = P, 
i Z Z 
qq -1") (p" ~1*) 
g, - U0"), p= 2D), 
(3.26) 
5 eM GHB) pip’ - FG" -2*) 
: 5! ; ‘ 5! : 
Hence Everett’s formula is given by 
Zz 
(q’ -1°) (g° - 1°) (g* -2°) 44 
Vy= QV +2 7 Aty, +02 Se Aa ype 
3! 5! (3.27) 
2 i 2 i 2 2 
PU ag PAP Pe Pa 
+ py, + rT A" ¥o + I Avy.y + 
where g=1-p. 


3.7.5 Relation between Bessel’s and Everett's Formulae 


These formulae are very closely related, and it is possible to deduce one from 
the other by a suitable rearrangement. To see this we start with Bessel’s formula 


~]) A*y, +A" —1)(p-1/2) 
Yp= Yor PAyy + PAE. ) ae Yo , Pip aa Ady, 


_ (P+) p(p-)(p-2) Aty 2 +A4y 
4! : 
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p(p-l) A*y_,+A7 yo , PApal) (p-1/2) 


=Yot PO - yt rr (A* yo- A*y1) 
4 4 
. ees a as ¥-1, 


expressing the odd order differences in terms of low even order differences. 
This gives on simplification 


PPD PD PPE) a2y ? 
4 6 7 


aes (ee), eae) 93 


= (- Pn +| 


a 2 Z Fr 
-1 1 
= ayy + LE 2, +:7:+ DH + PP ay fas 


which is Everett's formula truncated after second differences. Hence we 
have a result of practical importance that Everett's formula truncated after 
second differences is equivalent to Bessel’s formula truncated after third 
differences. In a similar way, Bessel’s formula may be deduced from Everett's. 


3.8 PRACTICAL INTERPOLATION 


In the preceding sections, we have derived some interpolation formulae of 
preat practical importance. A natural question is: Which one of these formulae 
gives the most accurate result’ 

(i) [f interpolation is desired near the beginning or end of a table, there 
is no alternative to Newton's forward and backward difference formulae, 
simply because higher-order central differences do not exist at the beginning 
or end of a table of values. 

(ii) For interpolation near the middle of a table, Stirling's formula gives 
the most accurate result for -1/4s p<1/4, and Bessel’s formula is most 
efficient near p=1/2,say 1/45 p<3/4. But in the case where a series of 
calculations have to be made, it would be inconvenient to use both these 
formulae, and a choice must be made between them. The choice depends 
on the order of the highest difference that could be neglected so that 
contributions from it and further differences would be less than half a unit 
in the last decimal place. If this highest difference is of odd order. Stirling's 
formula is recommended; if tt is of even order, Bessel’s formula might be 
preferred. Estimation of the maximum value of a difference of any order in 
an interpolation formula is not difficult. Thus, in Stirling’s formula (3.21), 
the term containing the third differences, viz., 


p(p*-1) By, +A yp, 
6 2 
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may be neglected if its contribution to the interpolate is less than half a unit 

in the last place. This means that 

p(p* -1) A*y_)+4°y9 
6 2 


for all p in the range 0S psi. 
But the maximum value of p(p* -1)/6 is 0.064 and so we have 


1 
<3) 


which gives 


If we consider Bessel’s formula (3.24), the contribution from the term 
containing the third difference will be less than half a unit in the last place 
provided that 


ee= DEP) ay a 


6 


But the maximum value of 


P(p-1)(p-12) 
6 
is 0.008, and so |A? y_, |< 60. In other words, if we neglect the third 
differences, Bessel's formula is about seven times more accurate than Stirling's 
formula. If the third differences need to be retained (i.e. when they are 
more than 60 in magnitude), then Everett’s formula may be gainfully employed 
for the aforesaid reason, viz., Everett's formula with second differences is 
equivalent to Bessel’s formula with third differences. The following examples 
illustrate the use of the central difference formulae. 


Example 3.10 The following table gives the values of e* for certain equidistant 
values of x. Find the value of e* when x = 0.644, 


x yoe 
0.61 1.840431 
0.62 1.858928 
0.63 1.877610 
0.64 1.896481 
0.65 1.915541 
0.66 1.934792 
0.67 1.954237 
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The table of differences is 


x ye" A a® ae a* 

0.61 1.840431 
0.018497 

0.62 1.858928 0.000185 
0.018682 0.000004 

0.63. 1.877610 0.000189 —0.000004 
0.018871 0 

0.64 1.896481 0.000189 0.000002 
0.019060 0.000002 

0.65 1.915541 0.000191 0.000001 
0.019251 0.000003 

0.66 1.934792 0.000194 
0.019445 

O.67 1.054237 

Clearly, 
pe ees ag 
0.01 


The third difference contribution to both Stirling’s and Bessel’s formulae is 
negligible, and using Stirling's formula, we obtain 


0.018871 —=— +28 (0.000189) 


= 1,89648 1 + 0.0075862 + 0.00001512 


= 1.904082, 
whilst Bessel’s formula gives 


y(0.644) = 1.896481 + 0.4(0.019060 + 


y (0.644) = 1.896481+ 0.4 


0.4(0.4—1) 0.000189 + 0.000191 
2 


= 1.89648 1 + 0.0076240 — 0.0000228 


= 1.904082. 
Using Everett's formula, we find that 
(0.644) = 0.6(1.896481) + “© °"°-9 (0.000189) 
+0.401.915541)+ 40°89 (0.090191) 


= 1,1378886 — 0.000012096 + 0.7662164 —0.000010696 
= 1.904082. 


In all the above cases, the value obtained is correct to six decimal places. 
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It is known from algebra that the nth degree polynomial which passes 
through (+1) points is wigwe. Hence the various interpolation formulae 
derived here are actually only different forms of the same polynomial. It 
therefore follows that all the interpolation formulae should give the same functional 
value. This is illustrated in the above example where we found that the interpolated 
value of 0.644 is 1.904082 regardless of which formula is used. 


Example 3.1] From the table of Example 3.10, find the value of e* when 
x= 0.638, using Stirling’s and Bessel’s formulae. 

It was mentioned in Section 3.8 that Stirling’s formula gives the most 
accurate result for -1/4s ps1/4, and Bessel’s formula is most efficient for 
1/4< p <3/4. In order to use these formulae, we therefore, have to choose 
xp 30 that p satisfies the appropriate inequality. 

To use Stirling's formula, we choose x, =0.64 and x, = 0.638 so that 
p=-0.2. Hence, 

0.018871 + 0.019060 . 0.04 


y(0.638) = 1.896481=-0.2- a ————— (0.000189) 


= 1.896481 — 0.003793 1 + 0.0000038 


= 1.892692, 
which is correct to the last decimal place. 


For Bessel’s formula, we choose x) = 0.63, x, =0.638 so that p=0.8. 
Hence, we obtain 


¥ (0.638) = 1.877610 + 0.8(0.018871) + 


2SCS— 0.000188) 


= 1.877610+ 0.0150968 — 0.0000151 


= 1.892692, as before. 


Example 3.12 The values of x and e* are given in the following table. 
Find the value of e* when x =1.7475. 


x y=e" A Ae aa at 

1.72 0.1790661479 
=17817379 

1.73 0.17 72844100 177285 
-17640094 —1762 

1.74 0.1755204006 175523 13 
17464571 =1749 

1.75 0.1737 739435 173774 ed 
-17290797 ~1727 

1.76 0,1720448638 172047 15 
=17118750 -i7 12 

1.77 0. 1703329886 170335 
=16948415 

1.78 0.1686381473. 
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It should be noted that in writing the differences in the above table, the 
zeros between the decimal point and the first significant digit to its right are 
omitted. Thus, in the column of second differences, the number 173774 
should be taken as 0.0000173774 in the computations. 

To compute y(1.7475), we choose xp =1.74 and x, =1.7475 so that 
p=3/4, We shall obtain the solution by using both Bessel's and Everett's 
formulae. 

(i) If we use Bessel's formula, the third differences need to be taken 

into account since they exceed 60 units in magnitude. Hence Bessel’s 
formula gives 


y (1.7475) = 0.1755204006 ~= (0.0017464571 


ri (3/4(3/4 — 1) 0.0000173523 + 0.0000 173774 
2 2 


=0.1755204006 -0.00130984284 — 0.000001 63734 + 0.000000001 37 


=0.1742089218, correct to ten decimal places. 
(ii) On the other hand, if we use Everett's formula up to second differences 
only, we obtain 


) 0,0000175523) 


y(1.7475)=2(0 1755204006) + oe 


% >(0.1 737739435) + 2DON6 =) ae =“) 


(0.0000173774) 


= 0.043380 10015—0.00000068564 + 0.13033045764 — 0.00000095033 


=0.1742089218, as before. 


This example verifies the result of Section 3.7.5 that Everett’s formula 
turncated after second differences is equivalent to Bessel’s formula truncated 
after third differences. When the fourth difference contribution becomes 
significant (i.e. when they exceed 20 units in magnitude), Everett's formula 
will be easier to apply since it uses only the even order differences. 


3.9 INTERPOLATION WITH UNEVENLY SPACED POINTS 


In the preceding sections, we have derived interpolation formulae of utmost 
importance and discussed their practical use in some detail. But, as is well 
known, they possess the disadvantage of requiring the values of the independent 
variable to be equally spaced. It is therefore desirable to have interpolation 
formulae with unequally spaced values of the argument. We discuss, in the 
present section and the next, four such formulae: (i) Lagrange’s interpolation 
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formula which uses only the function values, (ii) Hermite’s interpolation 
formula which is similar to Lagrange’s formula, (iii) Newton's general 
interpolation formula which uses what are called divided differences and 
(iv) Aitken’s method of interpolation by iteration. 


3.9.1 Lagrange's Interpolation Formula 


Let y(x) be continuous and differentiable (nm + 1) times in the interval (a, 5). 
Given the (m+ 1) points (xg, ¥g).(%1) Ms +--+ (ps ¥a) Where the values of x 
need not necessarily be equally spaced, we wish to find a polynomial of 
degree n, say L(x), such that 


L(x) =y@;)=y,, i120, 1 ...,9 (3.28) 


Before deriving the general formula, we first consider a simpler case, 
viz., the equation of a straight line (a linear polynomial) passing through two 
points (xp, ¥p) and (x), ),). Such a polynomial, say L,(x), is easily seen to be 


I~ *F iL= 
L(x) =— yp + = y 
xy — *y 


* ~ 9 
=lp()¥o +4) 
l 
=D ay 629) 
i=Q 
where 
(x)= band A(x)=——. (3.30) 
From (3.30), it is seen that 
lo(%) =1, fy (x) = 0, h(xp)=0, fy Gy)=1. 
These relations can be expressed in a more convenient form as 
l, ifi=/ 
! = 3.31 
j(x,) ‘6 fis j. ( ) 
The J,() in (3.29) also have the property 
l = -_ 
Yi) = h(x) + h(x) sb + 8. (3.32) 
j=0 Hoy Xp 


Equation (3.29) is the Lagrange polynomial of degree one passing through 
two points (xp, yo) and (x), 4). In a similar way, the Lagrange polynomial 
of degree two passing through three points (xg. yo), (4,4) and (42, ¥2) 15 
written as 
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Fi 
L,(x) = 2. d(x) y; 


i=0 


_ &-4)G@-») ; (x— Xp) (x— Xz) Pe (¥ — x9) (x=) 
(9-1) (%o~¥2) (1%) 1-2) X90) 
(3.33) 
where the /({x) satisfy the conditions given in (3.31) and (3.32). 
To derive the general formula, let 
Ly(X) = dy + a,x + ax? ++ + a,x" (3.34) 


be the desired polynomial of the mth degree such that conditions (3.28) 
(called the interpolatory conditions) are satisfied, Substituting these conditions in 
(3.34), we obtain the system of equations 


Yo = Ay + ayXp +a_x4 +--+ Gy xG 
= Z r 
Vy = Oy +.4jX, tagxy +e +4,x; 


Vz = Oy +X, +aQxS +e +a,x4 (3.35) 


Yq = My + Xq + OAK GH OAK. 
The set of Eqs. (3.35) will have a solution if 


1 2 a 


% %m "" yg 
Z fi 
] x X, oc x 
ae ! +0. (3.36) 
1 x Xs <a 


The value of this determinant, called Fandermonde's determinant, is 


(xp — %) (%q — 2) --- Go — 2p) 4 — 22). 0 — ye — An) 
Eliminating ap,a,,..-.a, from Eqs. (3.34) and (3.35), we obtain 


L(t) to «s xt ow 


¥o %y salt) xa 
y ] x x? oan ty = 0, (3.37) 
Vn I Xn Fs Sa 
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which shows that £,(x) is a linear combination of yy, ¥,.¥),.... ¥,- Hence 
we write 


L(x= >) by. (3.38) 
i=0 
where /{x) are polynomials in x of degree n. Since L(x; )= y, for j=0, 1,2, 
+ 4, Eq. (3.32) gives 


L(x) =0 if i+; 
# 
ij(x)=1 for all ; 
which are the same as (3.31). Hence /{x) may be written as 


(X—Xq)(X—X)...(X—%_1) (X¥—Xj,)).. (¥— 4H, ) 
(4, = Xq}(%) = ¥1) +0 Ot = 1) Oy = gg) OH = Hy)’ 


i(x)= (3.39) 


which obviously satisfies the conditions (3.31). 
If we now set 


Rae (X) = (% — XQ) (X ~ Hy)... (H— ¥)_y) (CZ — H)) (X— 4j,))- (—,), (9.40) 
then 


nal (x, ) = S [Trot (*)h es, 


= (2X) — Ky) (Xj — yo Oy — Hg) OG — Xp) Hy) 41) 
so that (3.39) becomes 
Hye (2) 
L(x)=—_ml* (3.42) 
i) (x =X) ) Mya (4) 
Hence (3.38) gives 
(x) 
(x)= yo “au _y,, (3.43) 
mn 3 (x- x;) ai (X) i 


which is called Lagrange’s interpolation formula, The coefficients /{x), defined 
in (3.39), are called Lagrange interpolation coefficients. Interchanging x 
and » in (3.43) we obtain the formula 


Fn) 
L,(y)= ) ———-x, (3.44) 
Press Oy) | 
which is useful for inverse interpolation. 
It is trivial to show that the Lagrange interpolating polynomial is unique. 
To prove this, we assume the contrary. Let £,(x) be a polynomial, distinct 
from L,(x), of degree not exceeding m and such that 


L@)=y%. 0,12...” 
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Then the polynomial defined by M(x), where 
M(x) = Ly(x)~Ly(2) 
vanishes at the (n+ 1) points x,, 7=0, 1, ...,. Hence we have 
M,,(x) = 0, 
which shows that J.(0r) and L(x) are identical. 

A major advantage of this formula is that the coefficients in (3.44) are 
easily determined. Further, it is more general in that it is applicable to either 
equal or unequal intervals and the abscissae x), x,,....x, need not be in 
order. Using this formula it is, however, inconvenient to pass from one 
interpolation polynomial to another of degree one greater. 

The following examples illustrate the use of Lagrange’s formula. 


Example 3.13 Certain corresponding values of x and log), x are (300, 
2ATT1), (304, 2.4829), (305, 2.4843) and (307, 2.4871). Find log; 301. 


From formula (3.43), we obtain 
(-3) (—4) (-6) (1) (—4) (-6) 
(2,477 1) + ———_——— (2.4829 
(—4) (—5) cay‘ a” (4) (-1) = : 
(1) (-3) (-6) (1) (-3) (-4) 
(2.4843) + —————- (2.487 1 
OO Or n@a@ C8 
= 1.2739 + 4.9658 — 4.47174 0.7106 
= 2.4786. 
Example 3.14 If py, =4, yy =12, yg =19 and py, =7, find x. 
Using formula (3.44), we have 
oo (-5) (-12) y+ PE) 3), GE) (4) 
(-8) (-15) (8) (-7) (15) (7) 


logig 301 = 


The actual value is 2.0 since the above values were obtained from the 
polynomial y(x) =x? +3. 
Example 3.15 Find the Lagrange interpolating polynomial of degree 2 


approximating the function y = In x defined by the following table of values. 
Hence determine the value of In 2.7, 


x y=lnex 

ri 0.69315 
2.5 0.91629 
3.0 1.09861 
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We have 


_ (x=2.5)(x-3.0)_ 49 
2 dara Trae Qx* —1ix +15. 


Similarly, we find 
h(x) =-(4x? - 20x4+24) and = Iy(x)= 2x7 -9x+410. 
Hence 
Ly(x) = (2x? —1 Lx +15) (0.69315) — (4x* — 20x + 24) (0.91629) 
+ (2x? —9x 410) (1.09861) 
= —(,08164x7 + 0.81366x - 0.60761, 


which is the required quadratic polynomial. 
Putting x =2.7, in the above polynomial, we obtain 


In 2.7 © Lo(2.7) = -0.08164 (2.7)* + 0.81366 (2.7) — 0.60761 = 0.9941 164, 


Actual value of In 2.7 = 0.9932518, so that 
| Error | = 0.0008646. 


Example 3.16 The function p=sin x is tabulated below 


x y=sinx 
0 0 
m4 0.70711 
a2 1.0 


Using Lagrange’s interpolation formula, find the value of sin (7/6). 
We have 


iy (20/6 — 0) (20/6 — 2/2) (0.70711) + (20/6 — 0) (27/6— 27/4) 


§ (a4 -0)(x/4-a72) (w/2=0) (w/2=n/4) 


8 ] 
=—(0.70711)-— 
9S 9 


_ 4.65688 
9 


= 0.51743. 


Example 3.]7 Using Lagrange’s interpolation formula, find the form of the 
function v(x) from the following table 
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x ¥ 
0 =|2 
l 0 
3 12 
4 24 


Since y=0 when x=1, it follows that x-1 is a factor. Let y(x)=(x—-I)R(x). 
Then A({x)=y/(x-1). We now tabulate the values of x and R(x). 


x Rix) 
0 12 
3 6 
4 8 


Applying Lagrange’s formula to the above table, we find 


Rx) = E=DE-M qn), E-NE-N og), G-O&-3) 


8 
-3)(-4) (3-0)(3-4) * (4-0)(4~3)"> 
=(x—3)(x -—4) -— 2x (x-4)4+ 2x (x-3) 
= x* —$x 412. 


Hence the required polynomial approximation to y(x) is given by 
y(x) = (x -1) (x? —5x+12), 


3.9.2 Error in Lagrange’s Interpolation Formula 


Equation (3.7) can be used to estimate the error of the Lagrange interpolation 
formula for the class of functions which have continuous derivatives of 
order upto (n+1) on [a, 5]. We therefore have 


yx) = Ly(x) = Ry(x)= ZH yor, gece (3.45) 


(n+1)! 
and the quantity E,, where 
E,, =max|R,(x)| (3.46) 
{a, 6] 
may be taken as an estimate of error. Further, if we assume that 
yO < May. SESb (3.47) 


then 
M, 
n+l 
LS (n+ Dla me (x)| (3.48) 
The following examples illustrate the computation of the error. 
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Example 3.18 Estimate the error in the value of y obtained in example 3.15. 

Since y=Inx, we obtain y' =1/x, y"=-1/x" and y'" =2/x". It follows 
that p’"(€)=2/€°. Thus the continuity conditions on y(x) and its derivatives 
are satisfied in [2, 3]. Hence 


(x-2)(x—2.5) (4-3) 2 


R, (x)= ; ans ae 
But 
CS een 
e a g 
When x=2.7, we therefore obtain 
i R,(x)| < (2.7 Se ee 3) ; _ eat = 0.00175, 


which agrees with the actual error given in example 3.15. 
Example 3./9 Estimate the error in the solution computed in example 3.16. 
Since y(x)=sin x, we have 
y'(x)=cos x, y' (x)=— sin x, y"(x)=— cos x. 
Hence | y'"(E)] <1. 
When x= 7/6, 


(7/6—-0)(2/6-0/4)(r/6-n/2)| laa 
a | Fe = — = = 0.02992, 
IXGIs 6 66123 


which agrees with the actual error in the solution obtained in example 3.16. 


3.9.3 Hermite’s Interpolation Formula 


The interpolation formulae so far considered make use of only a certain 
number of function values. We now derive an interpolation formula in 
which both the function and its first derivative values are to be assigned at 
each point of interpolation. This is referred to as Hermite’s interpolation 
formula, The interpolation problem is then defined as follows: Given the set 


of data points (x,, y;, ¥;), i= 0, 1, ...,, it is required to determine a polynomial 
of the least degree, say H3,,,(x), such that 
Han. (%) = ¥; and = Aon) (*)) = 945 fe), i,.... 4, (3.49) 


where the primes denote differentiation with respect to x. The polynomial 
H3,,;(x) is called Hermite’s interpolation polynomial. We have here (2n + 2) 
conditions and therefore the number of coefficients to be determined is (2n + 2) 
and the degree of the polynomial is (2n + 1). In analogy with the Lagrange 
interpolation formula (3.43), we seek a representation of the form 
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Hapa()= >, u(y, + > ¥,0y}, (3.50) 
i=0 i=0 
where w,(x) and v,(x) are polynomials in x of degree (2n +1). Using conditions 
(3.49), we obtain 
ule=16 eats (x)=0, fi ll 
ed a -¢ ;, -) W(x)=0, for all i 
es , (3.51) 
P . l, if i=; 
=0, for all f; (x, )}= 
ui; (x) or all i vj(,) fs if ij 
Since u,(x) and v,(x) are polynomials in x of degree (2n+1), we write 


u(x)=A(x) QP and ¥(x)=B() QP, (3.52) 


where /;(x) are given by (3.42). It is easy to see that 4,(x) and 8,(x) are 
both linear functions in x. We therefore write 


u(x) = (ax + b,) oor and =v,(x)=(c,x+d,) The) (3.53) 
Using conditions (3.51) in (3.53), we obtain 


ath (3.54a) 
cx, +d, =0 
and 
a, + 2I'(x;)=0 (3.54b) 
c,; =1. 


From Eqs. (3.54), we deduce 


a, =—2i;(x)), 6, =1+2x, (x) (3.55) 
c= l, d, =—X;. 
Hence Eqs. (3.53) become 
uj (x) =[-2x U(x) +1 + 2x) GP 
=(1-2(¢-x) f()),@P (3.56a) 
and 
v(x) = (x- x) [,)P. (3.56b) 


Using the above expressions for w{x) and v(x) in (3.50), we obtain finally 
Hoy (X) = > (1-20 -x) FLOP» +) &- x) COP y, 3.57) 
i=0 i= 
which is the required Hermite interpolation formula. 
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The following examples demonstrate the application of Hermite’s formula. 


Example 3.20 Find the third-order Hermite polynomial passing through the 
points (x,, ym), £=0, 1. 


Putting n=1 in Hermite’s formula (3.57), we obtain 


Hy (x) =[1 - 2(x — x9) HC) lg OY 9 + [1-2 — 4) A EO », 


+(x=x9) MoO? yp + x=) TAGOP 3. (i) 
Since 
os od bie 
f(x an (x) 3 
me a ae 
where hy =X) —Xq. Hence 
| , 1 
Kh(xy=-— and f(xj=—. 
g(*) rm hy 


Then, (i) simplifies to 


2(x—xp) | (x -x)" 2(x, -— x) | (x-x) 
Hs )14—— |S 14h |e 
3x) c hy he wots hy he y 


| 


_ wit 
at Yo +(x -%) 


+(xX— Xq) 


Vis (ti) 


(x -—Xg y 
FJ 


which is the required Hermite formula. 


Example 3.21 Determine the Hermite polynomial of degree 5, which fits 
the following data and hence find an approximate value of In 2.7, 


x yelnx y =l/x 
2.0 0.69315 0.5 
aa 0.91629 0.4000 
3.0 1.09861 0.33333 


The polynomials /,(x) have already been computed in Example 3.15. These are 
p(x) = 2x7 -1x4+15, 9 h(x) =-(4x7- 20424), (x)= 2x" -9x +10. 
We therefore obtain 
in{x) =4x—-11, ji{x) = —8x + 20, i5(x)=4x-9. 
Hence 
lo{%q) = -3, f(x) = 0, h(x) =3 
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Equations (3.56) give 
tig (x) = (6x -11) (2x7 -11x+15)?, vp (x) = (x - 2) (2x7 -1 Le + 15)? 


uy (x) = (4x7 — 20x + 24)?, v(x) = (x - 2.5) (4x? - 20x + 24)’, 


ua (x) = (19 - 6x) (2x7 - 9x +10)", vo (x) = (x —3) (2x? -9x +10), 
Substituting these expressions in Eq. (3.57), we obtain the required Hermite 
polynomial 
Hg (x) = (6x —11) (2x? = 11x +15)" (0.69315) 

+ (4x* — 20x + 24)? (0.91629) 

+ (19 — 6x) (2x7 — 9x +10)7(1.09861) 

+ (x—2) (2x7 -1 1x 415)? (0.5) 

+ (x — 2.5) (4x* - 20x + 24)? (0.4) 

+ (x —3) (2x” -9x +10)? (0.33333). 
Putting x =2.7 and simplifying, we obtain 

In (2.7) # H5(2.7) = 0.993252, 


which is correct to six decimal places. This is therefore a more accurate 
result than that obtained by using the Lagrange interpolation formula. 


3.10 DIVIDED DIFFERENCES AND THEIR PROPERTIES 


The Lagrange interpolation formula, derived in Section 3.9.1, has the disadvantage 
that if another interpolation point were added, then the interpolation coefficients 
i{x) will have to be recomputed. We therefore seek an interpolation polynomial 
which has the property that a polynomial of higher degree may be derived 
from it by simply adding new terms. Newton's general interpolation formula 
is one such formula and it employs what are called divided differences. It is 
our principal purpose in this section to define such differences and discuss 
certain of their properties to obtain the basic formula due to Newton. 
Let (Xp, ¥o)> (X15 Fy) ---» (Xq_s¥_) be the given (+1) points. Then the 
divided differences of order 1, 2,..., are defined by the relations: 


[xq) x] = 21-2, 
x] — Xp 


= [x, *2] =[Xp, x] 
[os Xs Xz] rare (3.58) 


[yy X94 009 Xp) —[XQ, 4, ---1 Xe] 
Kn — “9 


[Xp Xs +0+s Xn] = 
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Even if the arguments are equal, the divided differences may still have a 
meaning. We then set x) =x) +2 so that 


[xp.%)]= lim [xg, x9 + €] 
eo 


= lim 20+ 8) 9) 
e+) & 


= y'(xo). if (x) is differentiable. 


Similarly, 
_¥ Go) 3.59 
‘loka Leask | aa ne) 
(r+1) arguments 
From (3.58), it is easy to see that 
—_— ¥ — 
[Xp. 4] ~-m —% [% + Xp). 
Again, 
ee es ae) es 
a Xz — Xp [2 os | =) 


Xz —~Xy |X ~ 4 4-H NX AX 
ees: eee eer | Seen 
(Xp -—%)(% -—%2) (4 — Xp) (4) -— ¥2) 
es (3.60) 
(%q — Xp) (42 — 4) 
Similarly it can be shown that 


Yo ri Mi 


gl pe aeg a) ER Wr Se La er aR TL 
xq» nl] (%q —%y)---(%g —2_) Cy Xp). Oy ey) 


ae | eres (3.61) 
(x, —Xq)-+ (Xn — & -) 
Hence the divided differences are symmetrical in their arguments. 


Now let the arguments be equally spaced so that x, — xy =x) —x, =" 
=X, —X,-) =f. Then we obtain 


Mi — Yo 


x = XQ 


[xp. 4) = == Ay (3.62) 
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[xo, N+ Xz] = Ly, X2)— [Xo 4] : Xa)- 1x %1] os 


(280). 1 aye tay 
Xa — Xq 2h 


h oh) oop 2! 
(3.63) 
and in general, 
Expr 4>---» Male A ¥o- (3.64) 
If the tabulated function is a polynomial of nth degree, then A" yy would be 
a constant and hence the nth divided difference would also be a constant. 


3.10.1 Newton's General Interpolation Formula 
We have, from the definition of divided differences, 


7s 
[x, X] = oh 
so that 
y= My + (2-—Xy) Lx, 2). (3.65) 
Again, 
[x, xp, ] = eeol Bo a] 
i | 
which gives 


[x Xo) =[%9. 4] + - x1) [x, Xys %]. 
Substituting this value of [x,x)] in (3.65), we obtain 
Y= Vo + (X— 4X9) [xq 4] + (4 — 2X9) (x — Le, Xp, %)- (3.66) 


But 
[x, Xa Ths x3] — [x, x9. 2] —[%, 1, 22] 
t= Ts 
and so 
[x5 Xp» 1] = (xp, x15 2] +(x — 22) [% x9. 41. 2] (3.67) 


Equation (3.66) now gives 
Y= Yo + (X— Xp) [%p. 4] + — 2X) @- 4) Pp. 4,92] 
+(x— 4%) (x— x) (X— 22) [x, 29, 4, 29 ]- (3.68) 
Proceeding in this way, we obtain 
¥= Yo +(X— Xo) [%q, 4] + (4 — Xy) (X—X) [X0, X22] 
+ (x — xq) (x —x)) (« — X9) [%p, 4,49.) 4°°° 


+(x — Xp) (x — 4) ...(8 — x, [%, Kg, Xp, Xp) (3.69) 
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This formula is called Newton's general interpolation formula with divided 
differences, the last term being the remainder term after (n +1) terms. 


Example 3.22 As our first example to illustrate the use of Newton's divided 
difference formula, we consider the data of Example 3.13. 
The divided difference table is 


x logo * 

300 2.4771 
0.00145 

304 2.4829 0.00001 
0.00140 

305 2.4843 0 
0.00140 

307 2.4871 


Hence Eq. (3.69) gives 
logyg 301 = 2.4771 + 0.00145 + (—3) (-0.00001) = 2.4786, as before. 


It is clear that the arithmetic in this method is much simpler when compared 
to that in Lagrange’s method. 


Example 3.23 Using the following table find f(x) as a polynomial in x. 


x f(x) 
=1 3 
0 ~6 
3 39 
b 822 
7 1611 


The divided difference table is 


x F(x) 
= 3 
-9 
0 7 6 
15 5 
a 39 at 1 
261 13 
Fs 822 132 
789 
7 1611 


Hence Eq. (3.69) gives 
F(x) =34(241)(-9) + x(e+ 1) (6)+4(4 + D(x —-3) (5) + x(x 41) (x—3) (x-6) 


=x) ~ 4x) 45x? -6, 
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3.10.2 Interpolation by Iteration 


Newton's general interpolation formula may be considered as one of a class 
of methods which generate successively higher-order interpolation formulae. 
We now describe another method of this class, due to A.C. Aitken, which 
has the advantage of being very easily programmed for a digital computer. 
Given the (mn +1) points (x9, vo), (4, 4)).-..(%,.¥,), Where the values 
of x need not necessarily be equally spaced, then to find the value of » 
corresponding to any given value of x we proceed iteratively as follows: 
obtain a first approximation to » by considering the first-two points only; 
then obtain its second approximation by considering the first-three points, 
and so on. We denote the different interpolation polynomials by A(x), with 
suitable subscripts, so that at the first stage of approximation, we have 


1 |¥o Xp —* 
Ag (x)= yo + (2—%p) [X%. 4] = (3.70) 
ol YO Xo) L¥ ps 4 ee Wx 
Similarly, we can form Ag>(x), Ag3(x), -.- 
Next we form Ag; by considering the first-three points: 
Aoia(x)= 5a | (3.71) 
Xq — x, | Aga (x) Xy-X 


Similarly we obtain Ag)3(x), Ag)4(x), etc. At the nth stage of approximation, 
we obtain 


Aoig---n (2) =—— 


Xq ~ Xy—-1 


Agia ei) Fn 
boiz. pcan) X_ x | 


The computations may conveniently be arranged as in Table 3.7 below: 


(3.72) 


Table 3.7 Aitken's Scheme 


x ¥ 
Xp ¥o 
Agr) 
My vi Agi2(x) 
Aga(x) So123(*) 
2 ¥2 Agia(x) Agiza4(*) 
Agg(*) Any24(x) 
*3 ¥3 Apia(*) 
Agalx) 
x4 ¥4 


A modification of this scheme, due to Neville, is given in Table 3.8. Neville’s 
scheme is particularly suited for iterated inverse interpolation. 
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Table 3.8 Neville’s Scheme 


Aga lx) 
Aqa(x) 
Aga (x) 


Agq(x) 


Agi2(%) 
Aqa3(X) 


Aaaq(*) 


Agiza(x) 
Agiaaa(x) 
Ayaaa(*) 


As an illustration of Aitken’s method, we consider, again, Example 3.22. 
Example 3.24 Aitken’s scheme is 


Ft 


00) 


a4. 


305 


a07 


loging ¥ 
2.4771 


2.4829 
2.4843 


2.4871 


2.47855 
2.47854 


2.47853 


Hence log), 301 = 2.4786, as before. 
An obvious advantage of Aitken’s method is that it gives a good idea 
af the accuracy of the result at any stage. 


3.11 INVERSE INTERPOLATION 


2.47858 
2.47860 
2.47857 


Given a set of values of x and y, the process of finding the value of x for 
a certain value of y is called inverse interpolation. When the values of x are 
at unequal intervals, the most obvious way of performing this process is by 
interchanging x and y in Lagrange’s or Aitken’s methods. Use of Lagrange’s 
formula was already illustrated in Example 3.14. We will now solve the 
same example by means of Aitken’s and Neville’s schemes. 

Aitken’s scheme (see Table 3.7) is 


¥ 


19 


1.750 


1.600 
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whereas Neville'’s scheme (see Table 3.8) gives 


¥ x 
4 I 
1,750 
12 3 1.857 
2.286 
19 4 


Hence both the schemes lead to the same result ultimately. In practice, 
however, Neville’s scheme should be preferred for the simple reason that in 
this scheme those points which are nearest to x, are used for interpolation 
at x=x,. It is, of course, important to remember that inverse interpolation 
is, in general, meaningful only if the function is single-valued in the interval. 

When the values of x are equally spaced, the method of successive 
approximations, described below, should be used. 


Method of successive approximations 
We start with Newton's forward difference formula [see Eq. (3.10), Section 3.6] 
written as 


Vy = = yo + whys + 4E=D D A2y, + ROOD a yy + (3.73) 


From this we obtain 


| -1)(u-2 
we | nM Lady HONOR? 4854 -.} (3.74) 


Neglecting the second and higher differences, we obtain the first approximation 
to uw and this we write as follows 


1 
= Ou 90) (3.75) 


Next, we obtain the second approximation to w by including the term containing 
the second differences, Thus, 


e—_| ye —- yma a D 42 (3.76 
a) Ayo E yo ~ |. ) 
where we have used the value of u, for u in the coefficient of A*y9. Similarly, 
we obtain 


n-|n- yy - Aaa? - BOO = 7s 039 | (3.77) 
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and so on. This process should be continued till two successive approximations 
to w agree with each other to the required accuracy, The method is illustrated 
by means of the following example. 


Example 3.25 Tabulate y= x for x= 2, 3, 4 and 5, and calculate the cube 
root of 10 correct to three decimal places. 


x yer A Ae A= 
2 3) 

19 
a er 18 

af 6 
4 b4 24 

61 
5 125 


Here y,, =10, yo =8, Avy =19, A* yp =18 and A’ yy =6, The successive 
approximations to w are therefore 


1 
=—(7)-0.1 
Mj 19° 
_1f, 0.1@.1-1) ,0.]_ 
Ws ak 5 as) 0.15 


0.15 (0.15 —1) (0.15 - 2) 


ity = 6 


12-85 OD as) 


9 6)|- 0.1332 


; Qe I)(0. 1332-2) 


_1 2 Sass 


i 5 _ 0.1552 (0:1552—1) 
19 


(6)|-0.1541 


1541(0.1541-1)(0.1541-2 
; age If ; MM ) 


We therefore take y= 0,154 correct to three decimal places. Hence the value 
of x (which corresponds to y=10), ie. the cube root of 10 is given by 
xq + uh = 2.154, 

This example demonstrates the relationship between the inverse 
interpolation and the solution of algebraic equations. 


=— =0. 1542. 
r | 5 


3.12 DOUBLE INTERPOLATION 


In the preceding sections we have derived interpolation formulae to approximate 
a function of a single variable. For a function of two or more variables, the 
formulae become complicated but a simpler procedure is to interpolate with 
respect to the first variable keeping the others constant, then interpolate 
with respect to the second variable, and so on. The method is illustrated 
below for a function of two variables. For a more efficient procedure for 
multivariate interpolation, see Section 3.15. 
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Example 3.26 The following table gives the values of z for different values 
of x and y. Find z when x=2.5 and y=1.5. 


x 
y 0 1 2 3 4 
0 0 1 4 9 16 
1 2 3 6 11 18 
2 6 7 10 15 22 
3 12 13 16 21 28 
4 18 19 22 27 a4 


We first interpolate with respect to x keeping y constant. For x=2.5, we 
obtain the following table using linear interpolation. 


fan deen ee 
0 6.5 
I 8.5 
2 12.5 
3 18.5 
4 24.5 


Now, we interpolate with respect to-y using linear interpolation once again. For 
y=1.5, we obtain 


7a bd t!25 _ 195 
2 

so that 2(2.5,1.5)=10.5. Actually, the tabulated function is z =x" 4 y? +¥ 

and hence 2(2.5, 1.5)=10.0, so that the computed value has an error of 5%. 


3.13 SPLINE INTERPOLATION 


We have so far discussed methods of finding an nth-order polynomial 
passing through (m +1) given data points. Because of round-off and systematic 
errors, these polynomials were found to give erroneous results in certain 
cases. This is particularly so when the function undergoes sudden changes 
in the vicinity of a point in its range. Further, it was found that a low order 
polynomial approximation in each subinterval provides a better approximation 
to the tabulated function than fitting a single high-order polynomial to the 
entire range. These connecting piecewise polynomials are called spline functions, 
named after the draftman’s device of using a thin flexible strip (called a 
spline) to draw a smooth curve through given points. The points at which 
two connecting splines meet are called Anots. The connecting polynomials 
could be of any degree and therefore we have different types of spline 
functions, viz., linear, quadratic, cubic, quintic, etc. Of all these, the cubic spline 
(spline of degree three or order four) has been found to be the most popular 
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in engineering applications. We shall, however, start with a discussion of 
linear and quadratic splines, since this would set the theme for the derivation 
of the governing equations of the cubic spline. 


3.13.1 Linear Splines 
Let the given data points be 
(x). ¥y)s 2=0, 1, 21-9 (3.73) 
where 
A=Xy <2 <x <r ex, =h 
and let 
hy = Xj — Nay f=1,2,...,7. (3.79) 


Further, let s,(x) be the spline of degree one defined in the interval [x,_,, x; ]. 
Obviously, s,(x) represents a straight line joining the points (x,_), ¥;_;) and 
(x;, ¥;) Hence, we write 


5;{(x)= y)_, +m,(x-x)_)), (3.80) 
where 
m, = 221. (3.81) 
¥j ~ *-1 


Setting /=1,2,...,m successively in (3.80), we obtain different splines of 
degree one valid in the subintervals | to », respectively. It is easily seen 
that s,(x) is continuous at both the end points, 


Example 3.27 Given the set of data points (1, —8) (2, —1) and (3,18) satisfying 
the function y = f(x), find the linear splines satisfying the given data. Determine 
the approximate values of y(2.5) and y'(2.0). 
Let the given points be 4(1,-8), 8(2,—-1) and C(3,18). Equation of AB is 
5,(x) =-8 + (x -1)7 = 7x —-15, 
and equation of BC is 
$4 (x) = —-1+(x-2)19 =19x -39. 
Since x= 2.5 belongs to the interval [2,3], we have 
(2.5) = 35 (2.5) = 19(2.5) —39 =8.5, 
and 
y'(2.0) = m, =19. 


lt is easy to check that the splines s,(x) are continuous in [],3] but their 
slopes are discontinuous. This is clearly a drawback of linear splines and 
therefore we next discuss quadratic splines which assume the continuity of 
the slopes in addition to that of the function. 
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3.13.2 Quadratic Splines 


With reference to the data points given in (3.78), let s,(x) be the quadratic 
spline approximating the function y= f(x) in the interval [x,_,,x,], where 
Xx; — Xj) = A,. Let s,(x) and s/(x) be continuous in [xp,x,] and let 


5)(%) = ¥%, 4=0, 1, 2,...,% (3.82) 
Since s,(x) is a quadratic in [x,_,,x;], it follows that s;(x) is a linear function 
and therefore we write 
l 
si(x) =, 16 — x)nny_, +(x —%_1)m,], (3.83) 
where 
m, = 3;(x;). (3.84) 
Integrating (3.83) with respect to x, we obtain 


2 
n=} Ss i eh xy mj reat m lec, (3.85) 


where c; are constants to be determined. Putting x=x,_, in (3.85), we get 


2 ] 4; 7 hy 
ej = ¥y-] FT ea = Hi-1 FT -1- 


h, 2 Fd 
Hence (3.85) becomes: 


= 2 : 
s(x) = 4-85 (x - x)" x) mi ental y | Yi + “tm. (3.86) 


In (3.86), the m, are still unknown. To determine the m,, we use the condition 
of continuity of the function since the first derivatives are already continuous. 
For the continuity of the function s,(x) at =x; we must have 


5,(X)—) = 5;4,05;+) (3.87) 
From (3.86), we obtain 


§;(%)-) = +m + Hy + him 


= toms +m; )+ ¥j-1- (3.88) 


Further, 


2 rt] 

l Xj. — (x-x,) 

5444 (X) = -§ Fry “F : PMs | +y, +78 FA; y 
Fis 2 2 
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and therefore 


suieyt) =a Mm, + Vy) + +e m, = yj. (3.89) 
Equality of (3.88) and (3.89) produces the recurrence relation 
2 
Hiy_y + My “7, = Vib i=l, oy veeg (3.90) 


for the spline first derivatives m,. Equations (3.90) constitute m equations in 
(a + 1) unknowns, Viz, mig, im,...,m,. Hence, we require one more condition 
to determine the m, uniquely. There are several ways of choosing this condition. 
One natural way is to choose s/(x,)=0, since the mechanical spline straightens 
out in the end intervals. Such a spline is called a natural spline. Differentiating 
(3.86) twice with respect to x, we obtain 


s(x) = om +m), 
or 
si(x)= im, = My). 
hy 
Hence, we have the additional condition as 


Hil = Mh . (3.9 l } 
Therefore, Eqs. (3.90) and (3.91) can be solved for m,, which when substituted 
in (3.86) gives the required quadratic spline 
Example 3.28 Determine the quadratic splines satisfying the data given in 
Example 3.27. Find also approximate values of (2.5) and y'(2.0). 
We haven = 2 andh=1. Equations (3,90) give 
Mg tm, =14 and om, +m, =38. 


Since my =m,, we obtain mg =m, =7, and m,=3i. 


Hence, Eq. (3.86) gives: 


2 z 
s(x) =- MAP 7) SV 31) -14 


aye att zk a2 £-(7)+ S(e-2' +2 
nh sais 


which is the spline in the interval [2, 3]. 
Hence, 
(2.5) @ 53(2.5)=5.5 and y'(2.0)* 7.0. 
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The quadratic spline s,(x) in the interval [1, 2] can be determined ina similar 
way. A straightforward way of deriving the quadratic splines is as follows: 
Since s(x) is a quadratic in (x,_),x,), We can write 


5(x) =a, +bx+¢,x", (3.92) 


where a,, 6; and c,are constants to be determined. Clearly, there are 3 
constants and therefore we require 3n conditions to determine them. These 
conditions are obtained by using the properties of the quadratic spline. Firstly, we 
use the condition that the spline passes through the interior points. This means 


$; (x) = a, +5,x, +e,x? i=1,2,...,.4-1. (3.93) 
Next, s(x) is continuous at x =x, This condition requires 
5,(%;—) = 544, (3; +). (3.94) 
Hence, we must have 
a, + bx, tex? =a) +B +e yyx7,  F=1,2,....n-1. (3.95) 


Again, sj(x) is continuous at x =x,;. This gives 
by + 2ex, = by) + 2e),1%), f=1,2,....4-1. (3.96) 


We thus have 3n—3 conditions and we require three more conditions. Since the 
spline passes through the end points also, we must have 


Yo =a + byxy +045 (3.97) 
and 
Voy = Op + ByXpq + OpXn- (3.98) 
Finally, for the natural spline, we have 
s}'(xg) = 0, (3.99)] 
and this gives 
e, =0. (3.100) 


We have thus a completed system of 3 equations in 37 unknowns. Although this 
system can certainly be solved, it is obviously more expensive and therefore 
this method is less preferred to the previous one. 

The discontinuity in the second derivatives is an obvious disadvantage 
of the quadratic splines and this drawback is removed in the cubic splines 
discussed below. 


3.14 CUBIC SPLINES 
We consider the same set of data points, viz., the data defined in (3.78), 


and let s(x) be the cubic spline defined in the interval [x,_),x,]. The conditions 
for the natural cubic spline are 


Presented By: http://www.ebooksuit.com 


Section 3,14: Cubic Splines 113 


(i) sx) is almost a cubic in each subinterval [x,,, x;], i=1, 2,...," 


Gi) 3,{x)=¥,, 1=9,1, 2,217, 
(ili) s,(x),s;(x) and s;(x) are continuous in [xp, x,J, and 


(iv) s;(xy)= s(x, )=0. 
To derive the governing equations of the cubic spline, we observe that the 
spline second derivatives must be linear. Hence, we have in [x,_),.,]: 


si(x) = ° [(x; — x) Mj + (x -— 4), (3.101) 


where A,;=x,;—x,. and s;(x,)=M, for alli. Obviously, the spline second 
derivatives are continuous. Integrating (3.101) twice with respect to x, we 
get 


§(x)= [eee =a) MeSH ny, |g dj(X-X)1), (3.102) 


where c, and d; are constants to be determined. 
Using conditions 5;(x;.;)= ¥)-; and s,(x,)=y,, we immediately obtain 


2 Z 
c ate as] and dj = i{-%a| (3.103) 


Substituting for c, and d; in (3.102), we obtain 


eh 2 
s(x oj ae ; 2 ats + Eo Mesos 


2 
{y MaJaa} ad 


In (3.104), the spline second derivatives, M,, are still not known. To determine 
them, we use the condition of continuity of s;(x). From (3.104), we obtain 
by differentiation: 


Z 
H(o)= jf AO, Mental yy, 


{o He Meso a } 
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Setting x =x, in the above, we obtain the left-hand derivative 


Z a 
5;(4-) = aM, > nto Mas iy = eM, 


= Oi mat EM + EM (i=1,2,....m). (3.105) 


To obtain the right-hand derivative, we need first to write down the equation 
of the cubic spline in the subinterval (x,, x,,)). We do this by setting i=i+1 
in Eq. (3.104) 


3 h? 
514) (4) = oe" my, M, + OP Mia , = “Em -x) 


hin 
+| Vial ge (x-x,) 5 (3.106) 


where A,.,=X,;.,—x; Differentiating (3.106) and setting x = x,, we obtain the 
right-hand derivative at x =x, 


sali) = 5 — Ott ~y)~“ist M,- ~ Ah (i=0,1,...,2-1). (3.107) 


Equality of (3.105) and (3.107) produces the recurrence relation 


l 
PM, +30 + gM, ELM 


= STM Yi Fie (i=1,2,...,2—1). (3.108) 
hig) h, 
For equal intervals, we have Ay=h,,,;= A and Eq. (3.108) simplifies to 


Mj1 + 4M; + Mix = 5 Vi —2y,+¥4)s (i=1, 2,...,2—-1). (3.109) 


The system of Eqs. (3.108) has some special significance. If Mp and M, are 
known, then the system can be written as 
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(My +My +hy My =6{ 22= 2-28) nt 


hyM, + 2(hy + hy )Ma + iM; =6 


ra 72) — 2 


i hy 
(3.110) 


hy My + 2(hy + hy )M; +hyM, se Vy 22 


bg-Myn + 20hyg + Ihg)M p-) ot oo ee Jer 


Equations (3.108) or (3.109) constitute a system of (m—1) equations and 
with the two conditions in (iv) for the natural spline, we have a complete 
system which can be solved for the Mj. Systems of the form (3.110) are 
called tridiagonal systems and in the Ch. 6, we shall describe an efficient and 
accurate method for solving them, When the Af; are known, Eq. (3.104) 
then gives the required cubic spline in the subinterval [x,_), x,]. Also, the y; can 
be obtained from Eqs. (3.105) and (3.107). 


Example 3.29 Determine the cubic splines satisfying the data of Example 3.27. 
Find also the approximate values of (2.5) and y‘(2.0). 


We have n= 2 and My =]; =0. Hence, the recurrence relation (3.109) 
gives M, =18. If s)(x) and s)(x) are, respectively, the cubic splines in the 
intervals l¢x¢52 and 23x43, we obtain 


s(x) = 3(x -1P - 82 - x) - 4(x-1) 
and 
54(x) = 3(3 —x)° + 22x - 48. 
We therefore have 


y(2.5) * 5,(2.5) = + 7=7.375 
and 


y (2.0) & 55 (2.0) = 13.0. 


It should be noted that the tabulated function is y=x° -9 and hence the 
exact values of y(2.5) and y'(2.0) are, respectively, 6.625 and 12.0. The 
convergence to the actual values, with the increase in the order of the 
spline, is clearly seen from examples 3.27, 3.28 and 3.29. In many applications, it 
will be convenient to work with the spline first derivatives. Denoting s;(x,) =m, 
and taking suitable combinations of Eqs. (3.105) and (3.107), we can derive 
the following relationship for the m, : 
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h 


| 


emp me te 


3 3 
= (Yi — Yi + ZS O4 - Mia) f=1,2,..,9-1. (3.111) 
i+] h; 


The cubic spline in (x,),x,) in terms of the m, is then given by 


s(x) = ima - x)°(x- 41) -my(e-3,4)°(%) -9} 
f 


- SOs =x) [2(r— x4) +h] + ¥) (2-44) 12) -2) +4]. GI) 


The above result can easily be derived using the Hermite interpolation formula 
given in section 3.9.3. 


For equally spaced knots, Eqs. (3.111) assume the simpler form: 
3 
m,_) + 4am, + mi) = 7 Oie —¥j-1) f=1,2,....98-1. (3.113) 


Equations (3.109) or (3.113) constitute (n—1) equations in (a+1) unknowns, 
Viz., Mg, im, ...,m, . Clearly, two further relations are required in order that 
a unique interpolating spline may be found. These conditions are called the 
end conditions and are discussed in detail in Kershaw [1971, 1972]. The 
following example demonstrates the improvement in accuracy of the cubic 
spline interpolates with successive interval halving. 
Example 3.30 Given the points (0,0), (7/2,1) and (#,0) satisfying the 
function y= sinx (0< xs), determine the value of y (1/6) using the cubic 
spline approximation. 

We have n =2 and A=./2, The recurrence relation for the spline second 
derivatives gives: 


Mo + 4M, + M3 = ox (0- 2+ Q)=-A5. 
x? 
For the natural spline, we have My a =0. Hence, we have 
12 
M, =a 
i 


In the interval [0, 2/2], the natural cubic spline is given by 


2x 3% 
s(x) = 2- ra -%) 
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(el) tom 


We next take A= 2/4, i.e. the data points are (0,0), (x/4,1//2), (/2,1), 
(3/4, 1/2) and (,0).In this case, the recurrence relation gives: 


Hence 


4Mf, +M, =-4.029 
M, +4], +M,; =-5.699 (i) 
M, +4M, =-4.029. 
since My = .M,4=0. Solving eqs. (i), we obtain 
M), = —0.7440, M, =-1.053, 
In 0sx<a/4, the cubic spline is given by 


M, = ~-0,7440. 


3,(x)= : [-0.1240(x*) + 0.7836(x)]. 


»( =) m5, 8 = 0.4998, 


This result shows that the cubic spline has produced a better approximation 
when the interval is halved. We finally consider values of y=sin.x in intervals 
of 10° from x=0 to @ and then interpolate for x =5°,15°,25°,35° and 
45°, using the natural cubic spline. The cubic spline values together with the 
exact values are given in the following table: 


Hence, 


y=sinx 
x (in degrees) Cubic spline values Exact values 
5 0.087 155743 0.087155530 
15 0.256819045 0.258818415 
25 0.422618262 0.422617233 
35 0.573576436 0.573575040 
45 O.707106781 0.707105059 


3.14.1 Minimizing Property of Cubic Splines 


We prove this property for the natural cubic spline. Let s (x) be the natural 
cubic spline interpolating the set of data points (x, ¥,), /=0, 1, 2,..., 4, where 
it is assumed that a = x) < x, <x, <+++<x,=5, Since s(x) 1s the natural cubic 
spline, we have s(x,)=)}, for all i and also 5"(xj))=s"(x,) =. 

Let z(x) be a function such that 2(x,)= y, forall i, and z(x), 2°(x), 2°(x) © 
are continuous in[a, 5]. Then the integral defined by 
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[= [vere (3.114) 


will be minimum if and only if z(x)=s(x). This means that s(x) is the 
smoothest function interpolating to the set of data points defined above, 
since the second derivative is a good approximation to the curvature of a 
curve. We write 


F] Fi 
(2"(x) I de = [s"(x) + 2"(x)—s"(x)P dx 
b B 
a | [s"(xy2dke + 2f s"(x)[2"(x) —8"(x)] dx 
‘ [e'@-s' Pa (3.115) 


Now, 


b — ft 
[,2°@le"@)-s" lar = D1 "soe" -s" hae 
= i=0 
a-l 
= &'@E'®)-s'@on 
i=) 


n=l Sal 
->f s!"(x)[2'(x)-s'(x))de. (3.116) 
jay “7 
The first term in (3.116) simplifies to 


5"(%q L2'_) — 3' I — 5" (xg) L2G) — 5°). 


Since s"(x,)=3°(x)) = 0, the above expression vanishes. Similarly, the second 
term in (3.116) is zero since s'"(x) has a constant value in each interval and 
s(x;)= 2(x,)= y; for all i. Hence, (3.115) becomes 


b b b 
[ [2"(x)P dx = j. [s"@)P de + i [2"(x)-s"(@Pde 3.117) 


[ewer [ e'wra (3.118) 


It follows that the integral 


a] 
is | [2"(x)P dx 
ad 
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will be minimum if and only if 
[, "@)-s"wPar=0, (3.119) 


which means that z“(x)=s"(x). Hence z(x)—s(x) is a polynomial in x of 
degree at most three in [a, b). But the difference z(x)— s(x) vanishes at the 
points i=0,1,2,..... It therefore follows that 


z(xj=s(x) a<xeb. 


3.14.2 Error in the Cubic Spline and Its Derivatives 


An estimation of error in the cubic spline and its derivatives will be useful 
in practical applications. 

The natural cubic spline yields a good approximation of a smooth function 
together with several derivatives, which is testified by the following theorem: 


Theorem 3.1 If ye c*[a,b}, Q=X% <x < 4X2 <-°<x, =, andif s(x) ts 
the natural cubic spline for which 


sx)=y,. 220,11 2,...,7 


then 
max |y(x)—s(x)|S—Mh?, (3.120) 
Mp S35 S5q 2 
where 
h=x4-% #20142...0 
and 


M =max|y"(x)|, Xp S XS X,- 


It 13 clear that as the interval length A becomes smaller the better approximation 
the spline gives. This is in contrast to the known peculiarities of Lagrange 
interpolation. The errors in the spline derivatives can be obtained by using 
the operator notation. To find the errors in the first derivatives, we start 
with the recurrence relation (3.113), viz., 


ry + din, + my) = (ea —¥-1)- 
That is, 
534) 448) +8 iat) == Obed Ye 
Using the operator notation, the above equation can be written as 


(E' +44) s(x) = =(E- Ey, (3.121) 


Presented By: http://www.ebooksuit.com 


120 CuaPTer 3: Interpolation 


Since Ewe”? where D=d/dx, Eq. (3.121) becomes 


(CM + 44 MD )o(x;) = (hem), (3.122) 
Now, 
2 4 5 
tango, wD? n’D* WD? 
2! 3! 4 5! 
and 
eM ype tO DY AD! DP 
23! 4! 5! 
Hence 
4.4 6 76 
ghD 4 e-AD 9 1D? AND A’D 
2 24-720 
and 
3 5n4 
«=f avo 4B +} 
6 ~—- 120 


Using the above expressions in (3.122), we obtain 


h’p* hp* ; 3 RD? kD? 
2{eP  . +4 eile AD + 6 + 120 tee |p 


+ 


+ 
6 120 


WD pn‘ pd 


The above equation simplifies to 


= 6(D +h’ D'6 + h°D'/120+-->) | _ D+h D6 +h* D°N20+-- 
6+h D +A DIN2+-- 14h? D?/6+h*'D1T2+-- 


| 
np AtpD AD? , a*p* 
=| O+—— 4 ———— 4. | | | we nes yy 
6 120 6 72 


-[ooE ME. (22 + ve ve 


5'(x;) 


yi 


6 120 6 


(Po wot) 
6 72 M 
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4 4 nA 4 
Brg a FMD | EM pt 
6 72 36 


A‘ D? hp? h'D 
ts -fi-22 MO oly 


ee ee ip Ap’ 6 A‘D 
err 9 _ “+ ofp eee ¥; 
6 36 ~=—«120 


s'(x;) = yj -— ity} I +O(h°). 


In a similar manner, we can derive the relations: 


(3.123) 


s")=¥"G)- SPY + hy" )+008) G.124) 


ssa 4) 45", =") +S Hy") +OH4), 
s"(5/4)— sa) =") Wy) +08") 
From (3.123) to (3-126), we obtain 
y'(x)) =s'(x,)+O(H) 
Y")=3") + Py" G+ 00) 
Yi") =F" 4) +5" )+ 00?) 


yx) = s{s"(x/+) ~ $'"(x,)+ OCH). 


(3.125) 


(3.126) 


(3.127) 


(3.128) 


(3.129) 


(3.130) 


Relations (3.127)43.130) demonstrate that we can approximate y’(x,), y"(x,) 
and y"(x,) more accurately than y'"(x,), and this fact will be useful in the 
solution of differential equations with given boundary conditions. The above 


relations are due to Curtis and Powell [1967]. 
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3.15 SURFACE FITTING BY CUBIC SPLINES 


The cubic splines derived in the previous section can be extended to functions 
of two or more variables. We derive the formulae for functions of two 
variables, the extension to higher dimensions being straightforward.* Let 
LAx) be natural cubic splines which satisfy 


LAxj)=8y=1, jsi 
=0, j#i. 


(3.131) 


These splines bear the same relation to the general cubic spline as the 
Lagrange polynomials bear to the Lagrange interpolation polynomial. Due to 
this reason, we call them cardinal splines. Let s(x) be the natural cubic 
spline, in x; Sx<x,, corresponding to the set of data points (x,, y;), 
j=, 1, 2, ...,9. Then, Z,(x) are the cardinal splines corresponding to the 
set of data points (x,, 6, ,), where 4; ; is the Kronecker delta defined above. 
The cardinal splines are given by 


1| (x,-xy (x-2x,4)° h? 
£(x)= | d MM; j-1 $M, , +(5)-9{ 6 Br Mas 


vee-4p0(8,,-Mm,)} (3.132) 


where M, , = L,'(x,). It is easy to verify that (3.132) satisfies conditions 
(3.131). As in the case of general splines, the condition of continuity of the 
first derivatives leads to the recurrence relation 


a 
he 


In terms of the cardinal splines L,(x), the general spline s(x), in the 
interval x,_, SxSx,, can be written as 


M, j-1t 4M; 5p + Mi, ja => CG, ¢-1 ~ 28), 5 + 5), j41)- (3.133) 


s(x) = 7 Lx) ¥;. (3.134) 


i=Q 
where [,(x) are given by (3.132). 
Extension to functions of two variables is now quite straightforward. 
Let the values 


(x;,¥j)s i=0, 1, 2, sony 


of a function of two variables, z= f(x, y), be given at the n° data points 
arranged at the intersections of a rectangular mesh. The interpolation problem 


*See, Ichida and Kiyono [1974]. 
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now is to determine the value of z at an arbitrary point in the rectangular 
region. The cubic spline formula is given by 


Ai At 
s(x, y=), 2) Lj0) %, (3.135) 
i=0 s=0 
where L,(x) and L(y) are given by formulae of the type (3.132). The spline 
second derivatives, Mj, are calculated from the recurrence relation (3.133) 
by imposing the natural end conditions, M, 9=M, ,=9. 
The following examples demonstrate the use of the formulae derived 
above. 


Example 3.3] Using the data of Example 3.27, viz., (1,—-8), (2,—-1) and 
(3, 18), find the cardinal splines L,(x) and hence determine the general natural 
cubic spline in the interval 1< x2, 

For the interval lsxs2, we have j/=1. WithhA=1, and j=1, Eq. (3.132) 
gives: 


(x- 
6 


_ 3 
12) =2=2) a, 4 + y M; | +(2-2)(8,0- 2M, 0 | 
r(r-9 (65, tM, 


_@-1p 


M, | +(2-2)6, 9 +(%-1) [6,1-2M@.} (i) 


since Af, 9=0 for the natural cubic spline. 
Similarly, the recurrence relation (3.133), becomes: 


4M, | =6 (5, 9-26), +4), 2), 
from which we obtain 
3 3 
Mo. ~ 3° Mi) =-3, M3 | at 


Hence, (1) gives: 


3 
ta(s)= EF ( 3) -@-m+(-n(-F} =e Feed, (ii) 
Ly(x)=-5 x1 +3(e-D, (iii) 


n(x) =2(x-1) - 702-1) (iv) 


Hence, in l= x= 2, the general natural cubic spline is given by 
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2 
(x)=) wLi(2) 
i=0 


ab ea ees. 3 ype pe een} hr 
lie ly 51+ | +|3c 1) has I) Ic 1) 
+| Feet) -H¢e-p [as 


=3(x-1) +4x-12, 
which is the same as that obtained in Example 3.29. The next example 
demonstrates the use of cardinal splines in surface fitting. 


Example 3.32 The function z= f(x, y) satisfies the following data for 0 < x, 
y 2. Determine the natural cubic spline s(x, y) which approximates the 
above data and hence find the approximate value of (0.5, 0.5). 


= 
y Of 2 
0 12 9 
i: 23:2: 10 
29 10 17 


For determining 2(0.5,0.5), we need to obtain the natural cubic spline 
for the interval Osx, ysl. 


With A=1, j=1, we have 


—xy x 1 l 
L()=£ 2 Mio +, +0-2)(8,0-2Mi0 #48. 4M} 
2M, +(1-x) 5) 9 +3( 5. -1M,1} (i) 


since M; 9=0 for the natural cubic spline. Also, 
3 
M, = 3 1,0 — 26, , +6), 9). 


From eq. (i), we then obtain 
x 5x 
Lo(*) = ry = zi +1, 


h(x)=-3x" +3x, 
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Hence, in OS x, yS1, we have 


2 2 
5(x, y=y > L,(x) L(y) 2, j 


~ ire lat. + L(y)29,1 + £6(¥)20, 2] 
+4(x) Hg O)4, 0 +4021) + 409%,2] 
+£y(x)[L£)0)22,9 + G(y)22,1 + 402,21. 
Since x= y=0.5, the above equation gives: 
z(0.5, 0,5) = s(0.5, 0,5) 


= B (Bate txa-dx9]e 2 Bxaettna-2 x10) 
32 32 16 32 16\,32 16 32 
-2(Bx9+Hx10-3.x17) 
32\, 32 16 32 
= 0.875. 


The tabulated function is z=x°+ °+1 and therefore the exact value of 
z(0.5,0.5) is 1.25, which means that the above interpolated value has an 
error of 30%. 


EXERCISES 


3.1. Form a table of differences for the function f(x)=x° +5x—7 for 
x=-1, 0, 1, 2, 3, 4, 5. Continue the table to obtain f(6) and f(7). 


3.2. Evaluate 
(a) A*x? (b) A?(cos x) (c) A[(x+1l)(x+2)] 
(d) A(tan'x) = (e) ALS(=)e(=)]. 

3.3, Locate and correct the error in the following table of values: 


+ ld 
2.5 4.32 
3.0 4.83 
3.5 5.27 
4.0 5.47 
4.5 6.26 
5.0 6.79 
3.3 7.23 
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3.4, Prove the following: 
(a) uw, = Wy + Au,_> +Atu, 3 +--+ Ale, HAM Uy 
(b) A"y, = Vean —"CWeen-i °C Veen? Se {- 1)" Vy 
(c) wy +uy tertu, ="Cymy +"CyAuy tet AT ty, 


3.5. From the following table, find the number of students who obtained 
marks between 60 and 70: 


Marks obtained No. of students 


0-40 250 
40-60 120 
60-80 100 
80-100 70 

100-120 50 


3.6. In the following table, the values of y are consecutive terms of a 
series of which the number 31 is the 5th term. Find the first and 
the tenth terms of the series. Find also the polynomial which 
approximates these values: 


ap eet 


So =) Oo Uf le 
o 


3.7. From the following table of values of x and f(x), determine (i) f(0.23) 
and (ii) (0.29): 


x f@) 
0.20 1.6596 
0.22 1.6698 
0.24 1.6804 
0.26 1.6912 
0.28 1.7024 
0.30 1.7139 
3.8. Find the 7th term and the general term of the series 3, 9, 20, 38, 


65, ... 
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3.9. The following values are taken from the table of cubes: 


x 


6.1 
6.2 
6.3 
6.4 
6.5 
6.6 
6.7 


y=x 
226.98] 
238.328 
250.047 
262.144 
274.625 
287.496 
300.763 


Find (6.36) and (6.61). 
3.10. Define the operators, A,V,d and £, £' and show that 


(i) A= EV Gi) V=E'A 
(iii) E=l+A (iv) Ev =1-V 
(¥) Aye =V'Ve4r = OVeae, 2 (vi) AVy, = Vay, = 3", 


(vii) AC{)=Ont rede (Will), AC) =-Avy/ Oe re) 
3.11. Show that F=1+A and AsV(l-Vvy". Also, deduce that 1+ A 


=(£-1)V7" 


3.12. The population of a town in decennial census were as under. Estimate 
the population for the year 1955 


Year 


1921 
1931 
1941 
1951 
1961 


Population 
(in thousands) 
46 
66 
8] 
93 
101 


3.13. Find the missing term in the following table: 


x 


0 
l 
2 
3 
4 


¥ 
l 
3 
9 
P 


81 


Explain why the result differs from 3° = 27? 
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3.14, The probability integral 


has the following values: 


x p 
1.00 0.682689 
1.05 0.706282 
1.10 0.728668 
1.15 0.749856 
1.20 0.769861 
1,25 0.788700 


Calculate p for x =1.235. 
3.15. Prove the following relations where the operators have their usual 


meanings 

(i) #E=A? Gj) BM? = y-82 
(ii) 14+ 57u*? =(1+84/2)" (iv) wE=Ep 

(v) V=esE? (vi) A-V=é? 


(vil) g=cosh (u/2) where u=hD 


(viii) f'(x) = Hd f(x) - /6)ud? f(x) + (30) ud fx) 
3.16. The values of the elliptic integral 
wi2 
K(m)= i! (i—msin2@) "2 49 
i] 


for certain equidistant values of m are given below. Use Everett's 
or Bessel’s formula to determine A(0.25). 


mn Kim) 
0.20 1.659624 
0.22 1.669850 
0.24 1.680373 
0.26 1.691208 
0.28 1.702374 
0.30 1.713889 
3.17. From Bessel’s formula, derive the following formula for midway 


interpolation: 
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I Pre: 2 3 4 
wa=5Z00tW- FEA y1 +Ajo)+ seg A y-2 + AS j)— 


Also, deduce this formula from Everett's formula. 

3,18, State, without proof, Stirling’s formula for central interpolation and 
mention its limitations. 
From the following table of values of x and y =e", interpolate the 
value of y when x=1,91 


x y=e* 
L.7 5.4739 
1.8 6.0496 
1.9 6.6859 
2.10 7.3891 
Z.1 8.1662 
Zed 9.0250 


3.19. Use Stirling's formula to find 43; from the following table: 
Wop = 14.035, gg = 13.674, gg = 13.257, 
35 = 12.734, May = 12.089, ys = 11.309. 


3.20. From the following table, find y when x =1.45. 


x ¥ 
1.0 0.0 
1.2 -—0.112 
1.4 - 0.016 
1.6 0.336 
1.8 0.992 
2.0 2.0 
3.21. The following values of x and y are given. Find (0,543): 

x y(x) 
0.] 2.631 
0.2 3.328 
0.3 4,097 
0.4 4,944 
0.5 5.875 
0.6 6.896 
0.7 $.013 
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3.22. Using Gauss’s forward formula, find the value of (32) given that 
F(25) = 0.2707, (30) = 0.3027, 
F (35) = 0.3386, (40) = 0.3794, 
3.23. Using Gauss's backward formula, find the value of {12516 given that 
{12500 =111.803399, 12510 =111.848111, 


,/ 12520 =111.892806, «(12530 = 111.937483 
3.24, Evaluate sin (0.197) from the following table: 


x sin x 
0.15 0.14944 
0.17 0.16918 
0.19 0.18886 
0.21 0.20846 
0.23 0.22798 


3.25. Using Everett's formula, evaluate f(25) from the following table: 


x F(x) 
20 2854.0 
24 3162.0 
28 3544.0 
32 3992.0 

3.26. Given the table of values: 

x y=Jfx 
150 12,247 
152 12.329 
134 12.410 
156 12.490 


evaluate ,/155 using Lagrange’s interpolation formula. 
3.27. Show that 


> _ 8, 
(t—t,) 2'(t,) 


i=l 
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3.29, 
3.30. 


3.32. 


3.33. 


3.34. 


3.35. 
3.36. 


3.37. 
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If x(¢) is analytic inside the closed contour C and if f, i, f%,....¢, 
lie inside C, show that the remainder term in the error formula for 
polynomial interpolation can be written as 

a) ¢_x)_ 4, 


2m - (r-t)a(r) 


Show that } ju, (x)=! for all x. 


Applying Lagrange’s formula, find a cubic polynomial which approximates 
the following data: 


x vy) 

-2 ~12 
-1 -8 
2 3 

3 5 


. Using Lagrange’s formula, express the rational function 


3x7 + x41 
(x-1)(x-—2)(x-3) 


as a sum of partial fractions, 

[Hint: Let f(x)=3x*+x+1. Form a table of values of f(x) for 
x=1,2,3. Obtain the second-order Lagrange polynomial £,(x)]. 
[Stanton] 

Express the function (x? + x-3)(x" - 2x" —x+2) as a sum of partial 
fractions. 

Given the data points (1, —-3), (3, 9), (4, 30) and (6, 132) satisfying 
the function y= f(x), compute f(5) using Lagrange polynomials 
of orders | to 3. 

Establish Newton's divided-difference formula and give an estimate 
of the remainder term in terms of the appropriate derivative. 
Deduce Newton's forward and backward interpolation formulae as 
particular cases. 

lf ji (x) =1/x*, find the divided-differences [a, 5] and [a, 4, ec]. 
Given the set of tabulated points (1, —3), (3, 9), (4, 30) and (6, 132), 
obtain the value of y when x=2 using: 

(a) Newton’s divided-difference formulae of orders | to 3, and 
(b) Aitken’s method. 

Show that the nth divided-difference [x), x,,..., x, ] cam be expressed 
as the quotient of two determinants, shown as follows: 
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l ] ] | 
a a | * Xn 
2 2 Z 2 
ry 3 | x4 xn 
[X05 Xp. Xa) = ; 
i is i a 
yO J ¥2 ¥n 
] | 1 i] 
X% sa | a =n 
2 Zz Zz 2 
Xo X; %©, xp 


3.38. If the abscissae x,,{=0,1,...,9 are distinct and if y= f(x) is a 
times continuously differentiable, show that 


[Xs %j2-+-1 XI] = f-- fox +X be tl yXy ah... ty, 


where fg 20 and L/_y ¢, =1. 


3.39. Tabulate the function y=sin x for x=0 to 1.0 in steps of A=0.01. 
Find the error of linear interpolation in this table. 


3.40. Find the error of quadratic interpolation in the above example. 

3.41. Prove that the third divided difference of the function f(x)=1/x with 
arguments p,g,r,s is —l/(pqrs). 

3.42. If f(x)=1/x, prove that 


(-1)" 
ee oe 


(Xp. %4s---9 XJ) = 


3.43. Given the table of values 


x fx 


50 3.684 
32 3.732 
34 3.779 
56 3.825 


Use Lagrange’s formula to find x when 3{x = 3.756. 


Presented By: http://www.ebooksuit.com 


EXERCISES 133 


3.44. From the table of values 


% ¥ 
1.8 2.9422 
2.0 3.6269 
2.2 4.4571 
24 5.4662 
2.6 6.6947 


find x when y =5,0 using the method of successive approximations. 
3.45, From the following table of values, find x for which sinhx=5: 


bi sinh x 
2.2 4.457 
2.4 5.466 
2.6 6,695 
2.8 8.198 
3.0 10.018 


3.46. Develop a subprogram, in FORTRAN or C, to implement Lagrange 
interpolation and test it on the data of Problem 24. Compare it with 
the result obtained by using the MATLAB ‘polyfit function’ to fit 
a fifth-order polynomial. 


3.47, Reciprocal differences: The concept of reciprocal differences will 
be useful in determining a continued fraction approximation which 
agrees with a tabulated function /(x) at the set of points xp, 1, ...5 %q 


We define quantities P(X], PX»), Py [Xp » Xp .¥], Py L001 20], 5 
called the reciprocal differences in the following way: 


dolx] = f(x) 
r= i) = a —Xq 
fils ¥] ~ Pplx]- dole) £@)— FG) 
tolxp. x1.) = — 


Alx. x]- Al% *] 
and so on. Then following the procedure outlined in the derivation 
of Newton's divided-difference formula, we derive the general formula 
Be i 
Golx) = f(x) + fee 
rTP 4) Abo) Blo A 251 


which is the required continued fraction approximation to the given 
set of tabulated values. 
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3.48. 


3.49, 


3.50. 


3.31. 


3.52, 


3.53. 


CHAPTER 3: Interpolation 
Use the above method to obtain a continued fraction approximation 
to the set of points (1, 1) (2, 4),(3, 9) and (4, 16). 


Apply reciprocal differences to recover the function f(x) =1/(1+x*) 
from the following data: 


x f(x) 
0 l 

I 1/2 
2 5 

3 1/10 
4 WT 
5 1/26 


Using Hermite’s interpolation formula, estimate the value of In 3.2 
from the following table: 


x y=lInx y =1/x 
3.0 1.09861 0.33333 
35 1.25276 0.28571 
4.0 1.38629 0.25000 


Find the Hermite polynomial of the third degree approximating the 
function (x) such that p(xp))=1, »(x,)=0 and y'(xg) = y'(4) =0. 
Show that the error in Hermite’s formula is given by 


(noi Ol? ’In+2) 
YWe)— Hopi) = ye (), 


where (x) is assumed to have continuous derivatives of order 
(2n+2) and €=£(x) is in the interval determined by the points 


Ey Xps+-+y Xp 
The function yoxr49 is tabulated below: 


- Md 
3 36 
4 73 
5 134 


Predict the value of (4.5) by using quadratic and cubic splines and 
state the absolute error in each case. 

Fit a cubic spline to the function defined by the set of points given 
in the table: 
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hi B 


3.56. 


3.57, 


3.58. 
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x ye 
0.10 1.1052 
0.15 1.1618 
0.20 1.2214 
0.25 1.2840 
0.30 1.3499 


Use the end conditions: 

(i) My=My =0 

(ii) #'(0.10)= y'(0.10) and s'(0.30) = »'(0.30). 
Git) s"(0.10) = y"(0.10) and s"(0.30) = y"(0.30). 


Interpolate in each case for x= 0.12 and state which of the end 
conditions gives the best fit. 


Deduce the expression for the error in the spline second derivative: 
FF FF l i 
s"(x,) = ¥ ae + O(h*). 


Determine the cubic spline s(x) valid in the interval [x,_), x,] for 
the following data, given that s"(x,)=y"(x,) and s"(x,)=y¥'"(x,): 


(a) x y=uxink {b) x y=tanx 
6.2 11.3119 LJ 3.6021 
6.4 11,8803 1.4 5.7979 
6.6 12.4549 1.5 14,1014 


In the interval [x,,x,,;], the cubic spline s,(x) may be expressed as 
$(x) =a, +5 (x—x)+¢,(x-x,) +d(x—x,), i=, L....2-1 


Determine the constants a; 5, c, and d,, using the conditions for 
a natural cubic spline. 

Develop a subprogram to implement the natural cubic spline 
interpolation and test your program on the data of Problem 53. Also, 
use MATLAB spline function on the same data and compare the 
results. 


The following table gives the values of z = f(x, ») for different values 
of x and y. Use the methed of Section 3.12 to find z when x =2.5 
and y=1,.5, Compare your result with the actual value obtained 
from f(x, y)=x +p" +p, 
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eee Lae 
y 1 2 3 
13 6 11 
2 7 10 15 
3.13 16 21 


3.59. Repeat problem 58 using cardinal splines. 

3.60. Develop a subprogram to evaluate the cardinal splines L(x) and the 
general cubic spline s(x, y). Test your subprogram on the data of 
problem 58. 
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Least Squares, B-splines and 
Fourier Transforms 


4.1 INTRODUCTION 


In experimental work, we often encounter the problem of fitting a curve to 
data which are subject to errors. This is contrary to the case considered in 
the preceding chapter where we assumed that the data are free of errors. 
A common strategy for such cases is to derive an approximating function 
that broadly fits the general trend of the data without necessarily passing 
through the individual points, The curve drawn is such that the discrepancy 
between the data points and the curve is least. The method of least squares 
is most commonly applied in such cases and is described in the earlier 
sections of this chapter. 

Data fiting by means of polynomials has been considered, in the previous 
chapter, from the viewpoint of cubic splines. The resulting approximation, 
called the cubic spline approximation, suffers from the disadvantage of 
being a global approximation, which means that a change in one point 
affects the entire approximating curve. We describe, in the present chapter, 
a method based on basis splines, called B-splines, which possess a local 
character, viz., a change in one point introduces a change only in the 
immediate neighbourhood of that point. The B-spline method finds important 
applications in computer graphics and smoothing of data. The ‘B-spline and 
its computation’ will be discussed in Section 4.5. 

In the previous chapter, we concentrated on polynomial interpolation, i.e. 
interpolation based on a linear combination of the functions 1, x, x’, ..., x". 
On the other hand, trigonometric interpolation, i.e. interpolation based on 
trigonometric functions 1, cos x, cos 2x,..., cos mx, sin x, sin2x,..., sin, plays 

137 
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an important role in modelling oscillating or vibrating, systems. The Fourier 
series is a useful tool for dealing with periodic systems but for aperiodic 
waveforms, the Fourier integral or the Fourier transform is the primary tool 
available. Numerical methods for the computation of discrete ‘Fourier 
transforms’ are discussed in Section 4.6. 

We shall finally consider, in the concluding section, the representation 
of functions by Chebyshev polynomials as also the economization of power 
series. These are important from the standpoint of digital computation. 


4.2 LEAST-SQUARES CURVE FITTING PROCEDURES 


Usually a mathematical equation is fitted to experimental data by plotting the 
data on a graph paper and then passing a straight line through the data 
points. The method has the obvious drawback in that the straight line drawn 
may not be unique. The method of least squares is probably the most 
systematic procedure to fit a unique curve through given data points and is 
widely used in practical computations. It can also be easily implemented on 
a digital computer. 

Let the set of data points be (x,, »,),i=1,2,...,mm, and let the curve 
given by = f(x) be fitted to this data. At x=x,, the experimental (or 
observed) value of the ordinate is », and the corresponding value on the 
fitting curve is f(x,). If e; is the error of approximation at x=.x,, then we 
have 


é; = ¥) — f(x;). (4.1) 
If we write 
S=[y, - fO)P +b — SOP 42+ — SP 


settee een te?, (4.2) 


then the method of least squares consists in minimizing 5, i.e. the sum of 
the squares of the errors. In the following sections, we shall study the linear 
and nonlinear least squares fitting to given data (x,, y,), i=1,2,..., mt. 
4.2.1 Fitting a Straight Line 


Let ¥ =a) +a,x be the straight line to be fitted to the given data. Then, 
corresponding to Eq. (4.2) we have 


S =[y — (dp +, )]? + Ly — (ag + 422 )P + 2+ Le — (Gg + Xm). (4.3) 
For Sto be minimum, we have 


Sn 70-20) (aq +4; ))— 2[ 2 - (ag + 04 %9 )]—-+-— 2[¥q, Cg + 2) Xp, )] 


(4.4a) 
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= 0 = -2x, [yy ~ (ay + ax, )] -— 2x2 [yy — (ag + 2X2] 


2|2 


— 2Xm [Ym — (ay +a,X,,)). (4.4b) 
The above equations simplify to 


Mdg + G(X + Xy te + Xm) = I+ te im (4.5a) 
and 


ay (x, +9 tetae bay (xe +23 fete? Jam yy ta ho tet Kg  (4-5b) 


or, more compactly to 


May 10> x; ) yj (4.6a) 


jzl i=l 


and 


dy >} x, cay: x? -> XV (4.6b) 
f=] f=] ist 


Since the x, and y, are known quantities, Egs. (4.5) or (4.6), called the 
normal equations, can be solved for the two unknown ap and a). 

Differentiating Eqs. (4.4a) and (4.46) with respect to a, to a, respectively, 
we find that aS! dag and &* Si! 8a; will both be positive at the points a) 
and a,. Hence these values provide a minimum of S. 

Further, dividing Eqs. (4.6a) throughout by m, we obtain 

dy + ax =, 

where (x, ¥) is the centroid of the given data points. It follows that the 


fitted straight line passes through the centroid of the data points. The 
following example demonstrates the working of this method. 


Example 4.1 The table below gives the temperatures T (in °C) and lengths / 
(in mm) of a heated rod. If /=a,+a,T, find the best values for a, and a). 


rT (in °C) /(in mm) 


20 800.3 
30 $00.4 
40 800.6 
50 800.7 
60 800.9 
70 801.0 
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To use formulae (4.6), we require ET, 5/, 57? and 7], and these are 
computed as in the following table: 


T (in °C) ! (in mm) had Ti 
20 800.3 400 16006 
30 800.4 900 24012 
40 800.6 1600 s2024 
50 800.7 2500 40035 
60 800.9 3600 46054 
70 807.0 4900 56070 
270 4803.9 13900 216201 


Using formulae (4.6) we then obtain 
6p +270a, = 4803.9 and 270a) +13900a, = 216201, 
from which we get ag=800 and a, = 0.0146. 


Example 4.2 Certain experimental values of x and y are given below 


x tg 
0 -] 
2 5 
5 12 
7 20 


If »=ag +a,x, find approximate values of ag and a). As in the previous 
example, we form the following table of values: 


x ¥ x xy 
0 -1 0 0 
2 5 4 10 
5 12 25 60 
7 20 49 140 
14 36 78 210 


Formulae (4.6) give the two equations 


4ay +14aq,+36 and 14a) +78a, = 210. 
Solving the above two equations, we obtain a, =-1.1381 and a, = 2.8966. 
Using these values we obtain y(5) = 13.3449. 
It may be noted that the given table is obtained from the relation 
y =-1.0334 + 2.6222 so that the correct value of y(5) is 12.0776. 


4.2.2 Nonlinear Curve Fitting 


In this section, we consider a power function, a polynomial of the ath 
degree and an exponential function to fit the given data points 


(x). Vi iz=],...,m 
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Power function Let y= ax" be the function to be fitted to the given data, 
Taking logarithms of both sides, we obtain the relation 

log y=loga+c log x, (4.7) 
which is of the form ¥ =a) +a,¥, where Y=log y, ag =loga,a, =¢ and 
X =log x. Hence the procedure outlined in the previous section can be 


followed to evaluate a) and a,. Then a and ¢ can be calculated from the 
formulae a, =loga and c= aq). 


Polynomial of the nth degree Let the polynomial of the nth degree, viz., 
¥ =a) + x+aox" +--4+4,x" (4.8) 
be fitted to the data points (x,;, y,),i=1,2,....m. We then have 
S = [yy — (ay + a,x +--+ agxf)P +L — (ay + 04x +--+.) 


$+ [Vig — (Og + Xp) #28 Oy Xm II (4.9) 
Equating, as before, the first partial derivatives to zero and simplifying, we 
get the following normal equations 


efi mi fay | mi 
may +a) x +a, 9) x) tt and xP =D Yi 
i=t i=l 


jm] j=l 


m nt ba = 
ay) x +a, >) xp testa, pte aii (4.10) 


i=l im] | i=l 


m m m rn 
oS xP Sz og, FM aS aPy, 
i=] 


i=l i=] i=l 


These are (+1) equations in (n+1) unknowns and hence can be solved for 
dp, @),-..9@,- Equation (4.8) then gives the required polynomial of the mth degree. 
It should be noted that for large values of nm, the normal equations given 
by (4.10) are unstable (or ill-conditioned) with the result that roundoff errors 
in the data may cause large changes in the solution. Such systems occur, 
quite often, in practice and we shall! study their nature in a later chapter. It 
is sufficient to remark here that orthogonal polynomials are most suited to 
solve such systems and one particular form of these polynomials, the 
Chebyshev polynomials, will be discussed later in this chapter. 


Example 4.3 Fita polynomial of the second degree to the data points given 
in the following table 


ee ee ee 
0.0 1,0 
1.0 6.0 
2.0 17.0 
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In Eq. (4.10), we require the quantities Ex, Ex; , Ex], 2x), Dy, Ey, and 
Ex; y;- These are computed as in the following table: 


x y x x x! xy xy 
0 1 0 o 0 0 0 
1 6 1 1 1 6 6 
2 17 4 8 16 x 68 
3 24 5 9 17 40 74 


Using Eqs. (4.10), we now obtain the equations 
Jay + 3a, + 5a, = 24 
Jay + 5a, + 9a, = 40 
Say +9a, +1 7a, = 74. 


the solution to which is a) =1, a, =2 and a, =3. 
The required polynomial is then given by ¥ =1+2x+ 3x*. From the 
given data points, it is seen that this polynomial fitting is ‘exact’. 


Exponential function Let the curve 
yrage™ (4.11) 


be fitted to the given data. Then, as before, taking logarithms of both sides 
of (4.11), we get 


log y = log dy + a,x, (4.12) 
which can be written in the form 
Z= A+ By, 


where Z=log y, A=log ay and B=a,. The problem therefore reduces to 
finding a least-squares straight line through the given data. 


Example 4.4 Determine the constants a and 6 by the method of least 
squares such that y= qe" fits the following data 


x ¥ 

2 4.077 

4 11.084 

6 30.128 

8 81.897 

10 222.62 

The given relation is y = ae"™. Taking logarithms of both sides, we obtain 

In y=In a+ be. 


Setting In y=Y¥,x=X, In a=ay) and b=q,, the above relation takes 
the form y=ap) +a,x, which is a straight line. 
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The method of procedure is the same as in Section 4.2.1 and we form 
the following table: 


N=ux Yeliny x XY 
2 1.405 4 2.810 

4 2.405 16 9.620 

6 3.405 36 20.430 

8 4.405 64 35.240 
10 5.405 100 54.050 
30 17.025 220) 122.150 


Formulae (4.6) give 
Say +300, =17.025, 30a) +220a, = 122.150, 
which yield the solution: 
a, =0.405 and a, =0.5. 
Hence 
a=e™ = 94 1.499 and b=a,=0.5. 


4.2.3 Curve Fitting by a Sum of Exponentials 


A frequently encountered problem in engineering and physics is that of 
fitting a sum of exponentials of the form 


y= f(x)= Ae + Aye? 4504 Ae (4.13) 


to a set of data points, say (x), 4), 0%), ¥a), 0665 (Spe Bad 

In (4.13), we assume that n is known and A;, Ay, ..., dy. Ay, Age .ces Ay 
are to be determined. It can be seen that f(x) satisfies a differential equation 
of the type 


in A] H-2 
d’y d yay” 


ee tt any =O, (4.14) 
where the coefficients a,,a,,...,a, are presently unknown. 

A method suggested by Froberg consists in numerically evaluating the 
derivatives d"y/dx", d"' yide""', ... at the n data points and substituting them 
in (4.14), thus obtaining a system of nm linear equations for the » unknowns 
d),@),..-,d,, Which can then be solved. Again, it can be verified that 
Aye Age.ees dy are the roots of the algebraic equation 


A" + aa"! + aa"? +--+4, =0, (4.15) 


which when solved enables us to compute 4), A;,..., 4, from (4.135) by 
the method of least squares. An obvious disadvantage of the method is the 
numerical evaluation of the derivatives whose accuracy deteriorates with 
their increasing order and leading, therefore, to unreliable results, 
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We describe now a computational technique, due to Moore [1974], 
which leads to more reliable results. We demonstrate the method for the 
case n=2. 


Let the function to be fitted to a given data be of the form 
y= Ae" + Ae, (4.16) 
which satisfies a differential equation of the form 
dy _ 
allt Se mS (4.17) 
la 2 


dx* 


where the constants a, and a; have to be determined. Assuming that ‘a’ is 
the initial value of x, we obtain, by integrating (4.17) from ‘a’ to x, the 
following equation 


y'@)-y'(a) =a y(x)- aya) tay | y(x) dx, (4.18) 


where y'(x) denotes dy/dr. 
Integrating (4.18) again from a to x, we obtain 


Y(x)- W(a)- y'(@Xx-a)=a, f y(x)de-ay(x-a)y(a)+ay ff vox) de ae. 
(4.19) 
Using the formula 


= fe-o oa (4.20) 


& ioe, H 


wf $0). te 


Eq. (4.19) simplifies to 


y(x)- y(a)-(x- a)y'(a) = a, [ v2) de~ay(x-0) y(a) +a [e-oxoa. 
a a ror 
In order to use Eq. (4.21) to set up a linear system for a, and a, y'(a) 


should be eliminated and this is done in the following way. Choosing two 
data points x, and x, such that a—x, =x, —a, we obtain from (4.21) 


Wx )- y(a)—- (4 - a) y' (a) =a, toe) dx — a, (x, —a) Wa)+ a, { (x -1) (rd 


Y(%2)- a) - (2 -a)y'(a)=a, fro dx — a(x, -a) y(a)+a, f (x2—f) py) at. 
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On adding the above two equations and simplifying, we obtain 


p(x) + ¥(%2) - 2 ya) = a 


| yoyde+ | y(x)dx 


+a, } (1-1) pat + { (x) -1) roa (4.22) 


a ad 


Equation (4.22) can now be used to set up a linear system of equations for 
a, and a), and then we obtain A, and A, from the characteristic equation 


2 eaaes: (4.23) 


Finally, 4, and A; can be obtained by the method of least squares. The 
method of procedure is illustrated by the following example. 


Example 4.5 Fit a function of the form 


y= Ae™ + Aye™* (i) 
to the data given by 
LE ee ee eee 

1.0 1.54 1.5 2.35 
1.1 1.67 1.6 2.58 
12 1.81 1.7 2.83 
1.3 1.97 L.8 3.11 
1.4 2.15 


Choosing x, =1.0, a=1.2 and x, =1.4, Eq. (4.22) gives 


a 14. 
0.07 =a, | - | y(x)de+ | ylr) de 
1.0 [2 


L.2 14. 
+a (10-9 wi dt + (1.4 “aya 
1.0 12 


Evaluating the integrals by Simpson's rule and simplifying, the above equation 
becomes 


1.$la, +2.180a, = 2.10. (ii) 


Again, choosing x, =1.4, a=1.6 and x, =1.8, and evaluating the integrals 
in (4.22), as before, we obtain the equation 


2.884, +3.104a, = 3.00. (iii) 
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Solving (ii) and (iii), we obtain a, = 0.03204 and a, = 0.9364. Equation (4.23) 


now gives 
A* — 0.032044 — 0.9364 =0, 
from which we obtain 
A, =0.988=0.99 and A, =-0.96. 
Using the method of least squares, we obtain 
A,=0.499 and 4, =0.491. 


The above data was actually constructed from the function y=cosh x so 
that A, = 4, = 1/2, A, =1.0 and A, =-1.0. 


4.3 WEIGHTED LEAST SQUARES APPROXIMATION 


In the previous section, we have minimized the sum of squares of the 
errors. A more general approach is to minimize the weighted sum of the 
squares of the errors taken over all data points. If this sum is denoted by 
S, then instead of Eq. (4.2), we have 


S=W, 04 — £0 + D2 - £02)? ++ ha Le — FE 
=Wye; + Wye3 +--+ Wee. (4.24) 


In (4.24), the ’; are prescribed positive numbers and are called weights. A 
weight is prescribed according to the relative accuracy of a data point. If 
all the data points are accurate, we set W, =1 for all i. We consider again 
the linear and nonlinear cases below. 


4.3.1 Linear Weighted Least Squares Approximation 


Let ¥ =a) +a,x be the straight line to be fitted to the given data points, 
VIZ. (xq, Vevey (Xe Vor Phen 


S(aq,.a,)=¥) Wi; Ly, - (aq + x,)P-. (4.25) 
i=l 
For maxima or minima, we have 
BS _ 8 9 (4.26) 
Gay Ga, 
which give 
as fui! 
a— = 29) WL ~ (ay +4))1=0 (4.27) 
Gay f=] 
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as fii! 
a= 2 DH Ly (am +x) 4 =0. (4.28) 
Gay 


i=] 


Simplification yields the system of equations for ag and ay: 


if AT im 
a> Weta > Wx, => My; (4.29) 
im] i=l i=] 
and 
a: Wx, +a5, W, x; 7? Wx, (4.30) 
ial jz] 


which are the normal equations in this case and are solved to obtain a and 
a,. We consider Example 4.2 again to illustrate the use of weights. 


Example 4.6 Suppose that in the data of Example 4.2, the point (5, 12) 
is known to be more reliable than the others. Then we prescribe a weight 
(say, 10) corresponding to this point only and all other weights are taken 
as unity. The following table is then obtained. 


x y w Wx Wx? Wy Wy 

0 -i 1 0 0 -1 0 

2 5 1 2 4 5 10 

5 12 10 50 250 120 600 

7 20 1 7 49 20 140 

14 36 13 59 30g 144 7&0 

The normal Eqs. (4.29) and (4.30) then give 

l3ay + 59a, =144 (i) 
9aq + 303a, = 750. (il) 


Solution to eqs. (i) and (ii) gives 
@ =-1.349345 and a, = 2.73799. 
The ‘linear least squares approximation’ is therefore given by 
y =—1,349345 + 2.737992. 
We obtain 


¥(5.0) = 12.34061 = 12.34061, 
which is a better approximation than that obtained in Example 4.2. 
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Example 4.7 We consider Example 4.6 again with an increased weight, say 
100, corresponding to y(5.0). The following table is then obtained. 


x ¥ w Wie Wx? wy Way 
0 -1 1 0 4) -1 4] 
2 5 1 2 4 5 10 
5 12 100 §00 2500 1200 6000 
7 20 1 7 49 20 140 
14 36 103 509 2553 1224 6150 


The normal equations in this case are 
103a, + 509a, = 1224 (i) 

and 

509aq + 2553, = 6150. (ii) 
Solving the above equations, we obtain 

dy =-1.41258 and a, = 2.69056. 
The required ‘linear least squares approximation’. is therefore given by 
y =—1.41258 + 2.690562, 


and the value of y(5)=12.0402. 

It follows that the approximation becomes better when the weight is 
increased. 
4.3.2 Nonlinear Weighted Least Squares Approximation 


We now consider the least squares approximation of a set of m data points 
(x,, ¥)), 4=1,2,...,m, by a polynomial of degree n<m. Let 


Y= dp +ajx+apx* +---+a,x" (4.31) 
be fitted to the given data points. We then have 


S (Qs Oy +5 Og) = >, Hj LY — (aq + ym He +G,x/)P. (4.32) 


i=l] 
If a minimum occurs at (ap, a,,...,4,), then we have 
oS _ oS _ oS aS _o. (4.33) 
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These conditions yield the normal equations 


if it nt Lill 
ag J) W +0, | Wit ta, Wine = >) Wy 
i=l i=l 


iml im] 


ur) fis} Fl 
ay > e Wix, +>) Wx; ++ dy > Wy => Wy (4.34) 


f=] {= 


— 

~ 
N 

= 


ffi nt ' ffi 3 i) 
Tt i+ Li f 
dy Y Wixi +a, Wixp +a, d Wix;"= dD) Wix?y,- 
i=! i=l f=h i=l 


Equations (4.34) are (n+1) equations in (m+1) unknowns ao, ay,...,a,,. If 
the x, are distinct with mn < m, then the equations possess a ‘unique’ solution. 


4.4 METHOD OF LEAST SQUARES FOR CONTINUOUS FUNCTIONS 


In the previous sections, we considered the least squares approximations of 
discrete data. We shall, in the present section, discuss the least squares 
approximation of a continuous function on [a@, 4]. The summations in the 
normal equations are now replaced by definite tntegrals. Let 


p(X) = dy tax t agx? ++ a,x" (4.35) 
be chosen to minimize 


b 
S (aq, Qj -.+9 Oy) = [we [y(x) = (ay + ax +--- + a,x" )P de. (4.36) 


The necessary conditions for a minimum are given by 


SS Bg (4.37) 


which yield 
b 
—2 [ ¥@DE)-C@ + ayX+ ax” + tax") dx =0 


b 
-2 | WDC) -(ay + x4 ax teeta) x dea 0 


b (4.38) 
—2 [ HL) — (ag + ay + ax” ree. a,x" )] x" dx=0 
f? 


b : 
-2 | ¥@D@)-@ + X+ Gx" 4+ 4a, x "hx" de = 0. 


a 
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Rearrangement of terms in (4.38) gives the system 
4 


i] Fa] b 

ay | W(x)dx + a, | xW (x) dx +---+a, | x" W(x) de = | W(x) v(x) de 
a a ad a 
b b 6 bh 

ay [xW'(x)de+a, fer) dt +---+a, [ett () de= fe W(x) p(x) de 


a a a a 


b b b b 
Q% [e7@d&+a, Je") des--+a, [xP"W(x) de= fr") y(x}de. 


(4.39) 
The system in (4.39) comprises (+1) normal equations in (n +1) unknowns, 
VIZ. ap, 4),42,-..,4, and they always possess a ‘unique’ solution. 


Example 4.8 Construct a least squares quadratic approximation to the 

function y(x)=sin x on (0, 2/2] with respect to the weight function W’(x)=1. 
Let 

Yrap + GX + ayx* (i) 

be the required quadratic approximation. Then using (4.39), we obtain the 
system 


afd ma nfl ni? 
a [ deta, [xdc+a, [ Pde = [ sin xd 
0 0 0 0 
nia aa aia ail 
ay [ xde+o [2 ax + ay | Par= J x sin x ad (it) 
i] 0 0 
mi ni mz aid 
ay J x? de+a, [Para [ a= j x? sin x dr 
0 O 0 i] 
Simplifying (ii), we obtain 
x. ® oa a 
ar eer ery 
Z 3 4 
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whose solution is 


_ 18 96 _ 480 
” ce 
_ _ 144 x 1344 4760 (iii) 
x a x 
240 2880 11520 
aS a et 
# iw x 


The required quadratic approximation to y=sin x on [0, a/2] is then given 
by (i) and (tii), 
As a check, we obtain, at x = 7/4, 


sin x oe -° aa = ().706167587, 
T ir 


The true value of sin (77/4) =0.707106781, so that the error in the above 
solution is 0.000939194, 


4.4.1 Orthogonal Polynomials 


In the previous section, we have seen that the method of determining a least 
square approximation to a continuous function gives satisfactory results. 
However, this method possesses the disadvantage of solving a large linear 
system of equations, Besides, such a system may exhibit a peculiar tendency 
called ill-conditioning, which means that small change in any of its parameters 
introduces large errors in the solution—the degree of i//-conditioning creasing 
with the order of the system. Hence, alternative methods of solving the 
aforesaid least-squares problem have gained importance, and of these the 
method that employs ‘orthogonal polynomials’ is currently in use. This method 
possessess the great advantage that it does not require a linear system to 
be solved and is described below. 


We choose the approximation in the form: 


¥(x) = ag fo(x) +a, A(x) +--+, f,(%), (4.40) 


where /,;(x) is a polynomial in x of degree j. Then, we write 
b 

5 (Gg,d,-.-+4_)= [ P@OC)-lmh+qf@)+-+4,f 0) 7 dr. (4.41) 
a 


For S to be minimum, we must have 
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b 
= m= —2 J W(x) (W(x) Lag Jo(x) + yf) +--+ a, FCO} fol) ae 


b 
a J 7) 0C)-Lagfol2)+ afi @+-+ aA VAG) 4 | 64 4 


vt nt COU A(X) de 


Es 
— 


b 
S 20-2 |W) (e)-Lagfol2)+a,fi)+ 


The normal equations are now given by 


b b b 
ay | W(x) fg) dere, [WE fo@)A@ad&+--+ 0, | Hf) fi@ax 


b 
= | W(x) We) folx) & 
b b b 
ay [ WDA) foaddeta, | WEIS) @)de+--+a, | W(x) A) flee 


6 
= | HOW fa 
6 b 5 
aay | WC) f(x) folx) de + ay [ WR) fy (2) Ai) det ay [CD Fp as 


b 
= [ We) ve) f,@) ae 
(4.43) 


The above system can be written more simply as 


b b 
ay { W(x) folx) fy(x)ebe+a, [ WX) i) fy @)de+-— 


5 b 
+0, [WOL@OL@d= (WE) yd, j= 0,14 2-50 
. (4.44) 
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In (4.43), we find products of the type /,(*) f,(*) in the integrands, and 
if we assume that 


; 0, pq 
J PD plage) a= | W(x) f p(x) de p=q ee 
a P F ' 


then the system (4.43) reduces to 


P h 
ay j W(x) fo (x) de = { W(x) v(x) fox) de 
a a 


'r] b 
On J W(x) f2(x) de = [we yx) f(x) de. 
a a 


From the above, we obtain 


b 
[ 7) WO) Fe 
a. = j=, l, Z, soup Pie (4.46) 


b 
[Ho sod 


Substitution of ay, a,,...,@, in (4.40) then yields the required least squares 
approximation, but the functions fo(x), A(x).-...4,(*) are still not known. 
The (x), which are polynomials in x satisfying the conditions (4.45), are 
called orthogonal polynomials and are said to be orthogonal with respect to 
the weight function W(x). They play an important role in numerical analysis 
and a few of them are listed below in Table 4.1. 


Table 4.1. Orthogonal Polynomials* 


Narre f(x) Interval Wix) 

Jacobi Pie Pix) [-1, 1] (1- x)" (14+ x)* (a, #>-1) 
Chebyshev Thl*) [-1,1] (1- eye 

(first kind) 

Chebyshev Up(x) [1,1] (1— x22 
(second kind) 

Legendre P(x) (-1, 1} ] 

Laguerre Lal) [0, cx) e* 

Hermite H,(x) (-=, =) a 


*For more details conceming orthogonal polynomials, see Abramovitz and Stegun 
[1965]. 
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A brief discussion of some important properties of the Chebyshev polynomials 
T(x) and their usefulness in the approximation of functions will be given 
in a later section of this chapter. We now return to our discussion of the 
problem of determining the least squares approximation. As we noted earlier, 
the functions f(x) are yet to be determined. These are obtained by using 
the ‘Gram—Schmidt orthogonalization process,’ which has important applications 
in numerical analysis. This process is described in the next section. 


4.4.2 Gram-Schmidt Orthogonalization Process 


Suppose that the orthogonal polynomial f(x), valid on the interval [a, 5], has 
the leading term x’. Then, starting with 


fg(*) = 1 (4.47) 
we find that the linear polynomial f,(x), with leading term x, can be written as 
A(eaxthyg fo, (4.48) 


where &, 9 is a constant to be determined. Since f,(x) and fy(x) are orthogonal, 
we have 


h a] b 
| W (x) fo(x) fix) de = 0 = | xW(x) fo(x) de + kg | W(x) f2(x) de 


using (4.45) and (4.48). From the above, we obtain 


b 
J x W(x) fg() a 
hig =-=_____—— (4.49) 
[ WOOFE @) de 
and Eq. (4.48) gives 
b 
x W(x) fo(x) de 
f(x) =x-5=—————_. 

[ HOSE Hae 


Now, the polynomial f(x), of degree 2 in x and with leading term x“, may 
be written as 


fylx)=x? + hoo folx)+ ky A) (4.50) 


where the constants 4,5 and &,, are to be determined by using the orthogonality 
conditions in (4.45). Since j5(x) is orthogonal to f(x), we have 
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b 
J WO) fox? + kyo fol) + ka lee =O. 


Since id W(x) fo(x) f(x) de =0, the above equaiion gives 


] b 
| x W(x) fo(x)de { x? W(x) dx 


b 
| W(x) fo (x)de | W(x) de 


ad 


Again, since f5(x) ts orthogonal to f(x), we have 


ib 
J WOAOL? +b of) + hay AC) ae = 0. 


Using the condition that fe W(x) fy(x) A(x) de =0, the above yields 
b 
[ PW de 
ky i= Sia SA (4.52) 


J W(x) f(x) dx 


Since ky 9 and k,, are known, Eq. (4.50) determines f(x). Proceeding in 
this way, the method can be generalized and we write 
FQ sx! +k, ofp) + ky AQ) tt ky af) 1@. (4,53) 


where the constants Ki are 50 chosen that JAX) is orthogonal to 
fol), AG) 5 fae). These conditions yield 


‘a 
[Wo f(x) de 
Sh (4.54) 


; ; 
[ WOOF? 0) dx 

a 

Since the a, and jx) in (4.40) are known, the approximation F(x) can now 
be determined. The following example illustrates the method of procedure. 


Example 4.9 Obtain the first-four orthogonal polynomials f,(x) on [=1, 1] 
with respect to the weight function W’(x)=1. 


Let fo(x)=1. Then Eq. (4.49) gives 
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x dt 


| ey 


We then obtain from Eq. (4.48), f(x) =x. Equations (4.51) and (4.52) give 
respectively 


f x dk 
ky 9 =- =-> 
| dx 
and : 
f xx de 
k= ~ Fo =0. 
J x*de 
=I 


Then Eq. (4.50) yields f,(x) =x? -1/3. 
In a similar manner, we obtain 


[ Pax 
k3,9 =-—— =, 
[ a 
=] 
I 
a 
“i-- =~ 
[ ae 


l 
| Ox -V3) dk 
ky > = - = 0. 


i) (x7 = 1/3) de 
a 
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It is easily verified that 
3 
A(x=x -3= 


Thus the required orthogonal polynomials are 1, x,x* —1/3 and 2° —(3/5)x. 
These polynomials are called Legendre polynomials and are usually denoted 
by (x). [t is easy to verify that these polynomials satisfy the orthogonal 
property (4.45), An important application of Legendre polynomials occurs 
in numerical quadrature (see Chapter 5). 


4.5 CUBIC B-SPLINES 


We have seen that a curve passing through a given set of data points must 
be dependent on some interpolation formula or an approximating function to 
establish its relationship with the given data. The interpolation formulae 
(including the cubic spline formula) discussed so far are ‘global in nature’, 
since they do not permit local changes in the data or curve. 

The B-splines are ‘non-global’. These are basis functions. This basis 
allows the degree of the resulting curve to be changed without any change 
in the data. The B-splines can be of any degree but, in computer graphics 
and other applications, B-splines of degree 2 or 3 are generally found to be 
sufficient. We therefore restrict our study to a discussion of cubic B-splines 
only. The cubic B-spline resembles the ordinary cubic spline, discussed in 
the previous chapter, in that a separate cubic is derived for each 
interval. Specifically, a cubic B-spline (or a #-spline of order four), denoted 
by #,(x), is a cubic spline with knots 4,4, &)4,4),4,.,; and k, which is 
zero everywhere except in the range &,_4 <x <<4,. In such a case, By,(x) is 
said to have a support [&,_,,4,]. It may be noted that a B-spline need not 
necessarily pass through any or all of the data points. Similarly, a B-spline 
of order mn (degree n—1), denoted by &.(x), is nonzero only in the 
range k_,<x<k,. ‘The theory for B-splines’ was first suggested by 
Schoenberg [1946] and a recurrence formula for its numerical computation 
was independently discovered by Cox [1972] and de Boor [1972]. The B- 
splines may be defined in several ways. A useful representation is that based 
on divided differences and this will be given in the next section. 

Let the set of data points be (x,, y,),f=1,2,....m, and aixsh, Let s(x) 
be the cubic spline with knots Rs Rs seis Bys where a<k, < ky cers, <b, 
Then the cubic spline By (x) with knots 4,,£2,4;,4, and &; must satisfy 
the following properties: 


(i) On each interval, the B-spline must be a polynomial of degree 3 or 
less, 


(ii) The B-spline and its first-two derivatives must be continuous over 
the entire curve, 
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(iii) By 5(x)>O0 inside [k,, ks], i-e., the B-spline is non-zero only over 

successive intervals, 

(iv) Bg s(x) is identically zero outside [k,, ks). 
For computational purposes, tt would be convenient to use the normalized 
B-splines, N4,(x), defined by 

Nail) = (ky — yg) Baj(*). (4.55) 

The sum of all the normalized B-splines in the given range is equal to 1. 


Figure 4.1 shows the graph of a cubic spline #45(x) with knots —2,-1, 0, 1 
and 2. 


=2 =1 0 1 2 
Figure 4.1 A typical cubic B-spline. 
In Fig. 4.1., By 2(x) has the following properties: 


(i) By, 2(-2) = By 9(2)=0 and BY”) =1 
(ii) By 9(-2) = By .(2)=0 (4.56) 
(iii) By’ 9(-2) = By 2(2)=0. 
Suppose now we have p knots denoted by k), kj,...,.4,. To define the full 


set of B-splines, it is necessary to introduce ‘eight additional knots,’ viz. 
ks, ka, hj, kos Kpsts Bpsas Kpsg and Kp.q. These are chosen such that 


kig<kqg<ky<kyp=a and b=kyyy <hpay <kpg3 <kpyg. (4.57) 


We now have (p+ 4) B-splines (of order 4) in the range aox<05, and then 
the general cubic spline s(x), with knots Bhai Bp has a wnrigque 
representation, in the range a<x< 5, of the form 


+4 
t= 5 aM (x), asess, (4.58) 


f=] 


where N, ,(x) are the normalized B-splines of order 4 and @, are constants 
to be determined. 
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45.1 Least-squares Solution 


To determine the coefficients @, in (4.58), we substitute x =x, and obtain 


+4 
5(x,) = }, ay aNq (x), rol, 2, ...,if. (4.59) 


i=] 
where p is chosen such that m>> p+ 4. In matrix form, Eqs. (4.59) can be 
written as 
Na=y, (4.60) 


where NV is an mx (p+ 4) band matrix and a,y are column vectors. The 
solution of Eqs. (4.59) may be obtained by solving the normal equations: 


N'Nae=N'y. ie. Nary (4.61) 


4.5.2 Representations of B-splines 


To define the cubic B-spline at x= &,, we first consider the five knots 
kya Kj-ysKjn2s%j-. and ky, where a<k,_4 and k,<b. We also define the 
function 


p. P? when P20 (4.62) 
0 when PSO. 


Then a unique representation of the cubic B-spline with knots 4)4 ...,4; is 
given by (Greville [1968]) 


c] i 
s(x) = By x)=) a,x! i > Byy(X— hy)? (4.63) 


j= m= i—4 


Unfortunately, the representation of the cubic spline, as given by (4.63), is 
computationally inefficient because of loss of accuracy through cancellation. 
Another representation of the B-spline, a traditional one, is through divided 
differences. The divided difference of fourth order of the function (£,, —x); 
with respect to the knots kj_4,4;3,4j)-2,4), and &, as arguments is denoted 
by [Aj_4.4j-354)-25%)-1) 4]. We then have (see Section 3.10) 


Ba (2) = [hj gs Kas Kia Ris HI 
_ (hy4 x + 
(ki-4 — Ring Meg — By Rg — a 4 - 4) 
: (k3-x)} 
(Kj_-s = Fe al MKi3 i Ky MR )_3 ~ Ay] ts -3 —k) 


oa 
ip 
(Fy — ing Wy — pg My — By KI ~%4) 
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Setting 
Wy (x) = (x— Kg) (2 - Kg) (4 - #2) (e-K)e-&,) (4.65) 
Eq. (4.64) can be expressed in the more compact form 
: (Kin -x)} 
B, {x)= ee, (4.66) 
4 i 2 yy hg) 


More generally, a B-spline of order m (degree n—1) is defined by 


i ik. - n=! 
By AX) = [Rien Reng pee Ki] = > a (4.67) 
maj Gf Som 
where 
Hy (X) = (8 — yi) (8 — Riga) --- (8 — A). (4.68) 
Recalling that 


ki » ki +H; Kj —[k » Aj 7 7 » A 
Ehicas Rigs Kis Brgy by Ja eaetina Bin Bil“ hin hin hia (4.69) 


Ky — Rig 


we obtain the relation 


By (x) — By jy) (4.70) 


By (x)= . 
al : k, —Kj4 


which is a récurrence relation. Similarly, for B-splines of order mn, we 
obtain the relation 


B,-1, p(x) - 5-1, i-1(*) (4.71) 


Si ky — kin 


for a recursive computation of the B-splines &, ,(x). Unfortunately, 
computational algorithms based on formula (4.70) or (4.71) have been 
found to be numerically unstable even for simple examples, However, algorithms 
based on a recurrence relation discovered independently by 
de Boor [1972] and Cox [1972] have been found to be both stable and efficient. 
This recurrence relation will be stated and illustrated in the next section. We 
conclude the present section with an example on the computation of cubic 
B-splines represented by (4.63). 


Example 4.30 Using the relation (4.63), determine the cubic B-spline s(x) 
with support [0,4] on the knots 0,1,2,3,4. Show further that such a 
representation will be unique if s(1) is specified. 


Since s(x) is a cubic B-spline over [0,4], we have 
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s(0) = (4) =0 
3'(0) = s'(4) =0 (i) 
s"(0) =s"(4) =0. 
Because of symmetry, we also have 
s'(2)=0 (ii) 
s(1) = (3). (ili) 
Now, on the interval [0,1], let s(x) be given by 
$(X) =cp +OX+ ex? + ox’. (iv) 
Since s(0)=0, we immediately have c, =0. 
Also, the conditions s’(0)=s"(0)=0 given c,=0 and c,=0. Hence (iv) 
becomes 
s(x) =¢x", (v) 
which is the cubic spline on [0, 1]. Obviously, c; = .s(1). Again, on the interval 
(0, 2), let s(x) be represented by 
s(x)= ex? + B(x- iy. (vi) 
where #4, is to be determined. Now, 
s(x) =3e,x* +38,(x-1)4. 
But s‘(2)=0. Hence we obtain 


I2c, +34, =0, 
which gives 
Bi, = —4ey. 
Substituting for §, in (vi), we obtain 
s(x) = 05x? - 4e3(x-1)2, (vii) 


which is the cubic B-spline valid in the interval [0,2]. Further, let s(x) be 
represented on [0,3] as 


s(x) = ¢4x° -— 4e;(x-1)3 + B,(x-2)3. (viii) 
But s(3)=s(1)=c;. Substitution in (viii) gives 
x= 27ey -32¢, + Bs, 
from which, we obtain 


Az = 6c}, 


Hence, (vill) becomes: 


s(x) = ¢3[x° - 4(x-1)} +6(x-2)3]. (ix) 
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Finally, let s(x) be represented on [0,4] as: 
s(x) =¢,[x° - 4 (x-1)8 +6(x-2)3]+ B(x-. (x) 

Since s(4)=0, eq. (x) gives 

0=c,(64 -108+48)+ A,, 
from which, we obtain 

A, =—4e;. 
Substitution in (x) gives the required B-spline as 
s(x) = ¢3x" — 4e3(x-1)3 + 6c3(x-—2)3 - 4e3(x-3)3, 

which will be unigue if c; =s(1) is specified. 


4.5.3 Computation of B-splines 


B-splines are most conveniently computed by the Cox—de Boor recurrence 
formula which is both stable and efficient. For B-splines of order n (degree 
n—1), the formula is given by 


(X= Fg By, ja 00) + CR; = ).B yy, 2) 


4.72 
kj a kin 


B, (x)= 


and holds for all values of x. For proof of this formula, see Cox [1972]. 
It is seen from (4.72) that the computation of 8, ; (x) for any value of x 
depends on the values of B,_; ,_,(x) and B,_, ;(x). Thus, to compute the cubic 
B-spline based on the knots, kj_4. &j_3, &-2, kj). &;, we need to compute, 
from left to right, the elements in the array: 


Byj-3 
Bo j_2 
Byj-2 Ba j-1 
Bo j-4 Bg i 
Byi-t Bg j 
Bp j 
By 


Further, advantage may be taken of the fact that some of the elements in 
the above array may vanish because of the properties of the B-spline. For 
example, if k,4 Sx<4_;, then using the relation 


1 
B= : fk.) sx<k 
i ee gate a Neate" (4.73) 


= 0, otherwise. 
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the above array takes the form: 


By j-3 
Bp j-2 
0 Bg, j- 
Q 84 
0 0 
0 
0 


The numerical computation of the B-splines will now, become more simpler. 
This is illustrated in the following example considered by Cox [1972]. 


Example 4.11 We consider knots 0, 1, 2,3, 4, 5, 6 and compute B-splines of 

order 6 (degree 5) at x=] and r=2, i.e. at the interior knots | and 2. 
Corresponding to x=1, we have ky sx<ky. 

Then 


and we need to compute 3, 7. Consequently, we need to compute only the 
elements in the following array: 


0 
Bog 
By 3 Bs 4 
O B45 
o 0 Bs 6 
0 0 Be 7 
0 0 0 
a) 0 
0 0 
ta) 
13) 
Using the Cox—de Boor formula, the values at x=1 of the above B-splines 
are given by: 


| =|], By 3 = ar By 4 = 1/6, By 5 = 1/24, By =), 


Bs, =W120, Bs7=0, By 7 =1/720=0,0013888..., 
which is the same as that value obtained by Cox. 
Again, corresponding to x=2, we have k, <x <k,. Hence, 


1 
Bi 4 = 


ky -k;, ~ 


and we require the value of By 7 at x=2. 
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The elements to be computed are given in the following array: 


0 
f) 
0 Ba 4 
Be 4 Bas 
Bi 4 B35 55.6 
B25 Bae Bg,7 
0 P36 Bs,7 
0 B47 
0 0 
0 
0 


The values obtained at x=2 using the Cox—de Boor formula are given by: 
By 4 = 12, By 5 =0, 
By 4 =1/6, By 5 =/6, B, ¢ =0, 
By 5 =1/6, Byg=V/24, By 7 =0, 
Bs ¢=11/120, Bs 7 = 1/120, 


Bg 7 = 26/720 = 0.0361111..., 
which is the same as that obtained by Cox. 


4.6 FOURIER APPROXIMATION 


The approximation of a function by means of Fourier series, i.e. by a series 
of sines and cosines, is found useful in applications involving oscillating or 
vibrating systems. Let the function f(r) be a periodic function with period 
T> 0, i.e. let 

F@+T)= fO, (4.74) 


where T is the smallest value satisfying Eq. (4.74). Then the Fourier series for 
F(t) is written as 


f@) =3+5) (a, cosa +b, sin 2), (4.75) 
n=] 


r 


where a, and 5, are real numbers independent of t and wy = 2a/T is called the 
fundamental frequency. The coefficients 27k/T,k=2,3,... are called 
harmonics. 


Integrating both the sides of (4.75) from 0 to 7, we obtain 


f r r 2irnt . Birr 
i i (dt = > [, dt + [, [ 2, cos = +6, sin = Ja or, 
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since 


Hence 


9 fT 
=— (4.76) 
a =7 i F(t) dt. 


Again, multiplying both the sides of (4.75) by cos(2ant/T) and then integrating 
from 0 to T, we get 


2;T Jani 
a, == i Fees . Jat, (4.77) 


r 
[ cos( ent sin ( a a =(). 
0 T r 


Finally, multiplying both the side of (4.75) by sin(2ant/T) and then integrating 
from 0 to 7, we obtain 


since 


2; | att 
=— —— dt. 4.78 
b, ={ fees Ja (4.78) 


Thus the coefficients ap, a, and 5, in the representation (4.75) are evaluated. 
If T=2, i.e. if f(r) is of period 27, the formula (4.76)}1(4.78) become: 


| FF 

ay =— [fede 

ee [ F(O)cos nt dt, (4.79) 
T #—7 


1 a 
b, =— i) F(e)sin nr at. 
oer 


The Fourier series becomes further simplified if f(r) is an even or odd 
function. If f(f) is even, then we have 


f= +) cos nt, 
n=] 
hie (4.80) 
2 
a, = 2 [/ reoe0s nf df, 


since b, =0. 
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Similarly, if f(t) is an odd function, then we have 


fo=-) b, sinnt, 


n=l 


(4.81) 


zt : 
b,, am { f(t)sinar de. 


since dp =a, =0. 
The formulae (4.75}(4.78) can be expressed in a different way. For this, 
the well-known relations are used: 


int j~int int —int 
cosmt=“<——"——._ and sinnt' == : (4.82) 
2 Zi 
Using (4.82), Eqs. (4.75}44.78) can be expressed as 
a 
= Yo Ayer, (4.83) 
p=-a 
where 
lia ~2mipt!T 
A, == fipe dt, =O, 1, 2,... 4.84 
PT Jrr - sla 


These formulae directly lead us to the discussion of Fourier transforms but, 
before this, we consider an illustrative example on Fourier series. 


Example 4,12 Find the Fourier series of the function defined by 


—l, -rer<0 
fin=4 0, r=0 
l, O<f<x. 


The graph of the given function is shown in Fig. 4.2 
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From the graph, it can be seen that f(r) is an odd function. Hence the Fourier 
series for /(f) contains only the coefficients h,. We therefore have 


F(Q)= > 5, sinne, 
fs ad 


where 


ee | " ¢@)sinnt dt 
Tv af 


2 iT 
-<| sinat dt, since f(fj=1 
T sO 


also, 
=—| -— cosa?l 
aT H a 


It follows that 


fa 
fipi= eee rt = (sin + sin at + a st+--| 
eel r 3 5 


4.6.1. The Fourier Transform 


In the preceding section, we considered the Fourier series for periodic 
functions. There exist, however, several functions which are not periodic. 
Similarly, we come across, in nature, many phenomena (for example, lightning) 
which are aperiodic, The story of such phenomena is of great importance to 
the engineer. In such cases, the Fourier transform is the applicable tool and 
this can be derived, from Eqs. (4.83) and (4.84), by making T approach infinity 
so that the function becomes aperiodic, When T + «, Eq. (4.84) can be written 
in the form 


Fiiay)= [seat (4.85) 
and is called the Fourier transform of f(t). Similarly, Eq. (4.83) is written as 
=< | Flare“ doy, (4.86) 

T J-0 


and is called the inverse Fourier transform of f(t). Equations (4.85) and (4.86) 
enable us to transform from time domain to frequency domain and from 
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frequency to time domain, respectively. Physically, F(ia,) represents the 
frequency content of the signal. In Eq. (4.85), the function /(f) is given in 
the continuous form which is rarely the case with a signal. In fact, the function 
(6) ts available only in a discrete form, i.e. (i) is specified only at the points 
f,,i=0,1,2,---, M-1, and At=7/N. Thus jf, denotes the value of f(r) at 
t,. Then, corresponding to Eq. (4.85) and (4.86), we have: 


=I 


F, = ih en2atlpIN p= 0, 1, 2. N= (4.87) 
k=0 
and 
i N=1 7 
N 7 
h=x DF em b] k=0, L, 2,...,V-1 (4.88) 
p=0 
Denoting 
Wy = go tatIN (4.89) 
Eqs. (4.87) and (4.88) become 
N-1 
Fe= > fey p=0,1,2,---,N-1 (4.90) 
k=) 
and 
1 V-l 
fe=3 Fon”, k=0,1,2,-,M-1 (4.91) 
pal) 


The above equations are, respectively, called the discrete Fourier transform 
(DFT) and the inverse DFT. They are the discrete analogues of Eqs. (4.85) 
and (4.86), respectively. The coefficients |F al form a periodic sequence 
when extended outside of the range p=0, 1, 2,...,N—1, and we have 


Faun =F, (4.92) 


A useful analysis that is important in the practical applications of Fourier 
transform (such as smoothing of noisy data) is called the power spectrum 
which is a plot of the power versus frequency. If f(r) is a discrete-time 
signal with period NV, then the power P is defined by the relations 


a ‘ if-l . 
Pay Ql “DI Fel (4.93) 


Therefore, the sequence 


re=lFiPs  =0,1.2,N~-1 (4.94) 
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is the distribution of power as a function of frequency and is called the 
power density spectrum of the periodic signal. The power spectrum is the 
plot of p, as a function of frequency mk. Since F,(k =0, 1, 2, ..., W—1) is 
also a periodic sequence with period N, it follows that the spectrum of 
F,(k =0, 1, 2,...,N—I) is also a periodic sequence with period N. Hence, 
any MV consecutive samples of the signal or its spectrum provide a complete 
description of the signal in the time or frequency domains. 


Example 4.13 Find the DFT of the sequence {1, |, 1, ..., 1} for &=0, 1, 2, 


41. 
We have 
N=1 
=> f WR, 7) 
kai) 
where 
i a oe (i) 


since f, =1 for all k=0, 1, ..., N—-1, it follows that 


NI N-1 
L= re p> eae 


k=0 
which is a geometric series of N terms with a common ratio of WP. We 
therefore have 
_1-( Ry" _1-R 


arr, p=0,1,2,..., N-1. (iii) 
1-Wy “Wy 


Fy 


for p= 0, it is seen that the ratio on the right side of (iii) is of the form 
0/0. Hence we obtain its limiting value as V by using L’Hospital’s rule. Similarly, 
for p=1, 2,..., N-l, the limiting value of F, is calculated and is found to 
be zero. We thus have 


i when p=0 
0, when p=1, 2, ...,.N-1. 


4.6.2 The Fast Fourier Transform 


The computation of DFT using Eg. (4.90) 1s inefficient because it does not 
make use of the symmetric and periodic properties of the factor Wy, viz., 


WEP) cw lPy® and WENT?) — ip whee (4.94) 


The direct use of Eq. (4.90) requires N” complex operations and also memory to 
store the values of f(f) and WP . As NV increases, the computation of DFT 
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demands very high memory requirements and becomes a complex and time 
—consuming process. 

A class of algorithms, called the fast Fourier transforms (FFT) , computes 
the DFT in an economic fashion using properties (4.94) and thereby reducing 
the number of operations to MN log,N. This means that, in terms of computing 
time and memory requirements, the FFT is far superior to the DFT. For 
example, for V = 50, the FFT requires about 250 complex operations compared 
to about 2500 complex operations required by the direct use of Eq. (4.90). 
This contrast, therefore, points to the importance of FFT algorithms. There 
exist several FFT algorithms and the basic idea behind all these is that a 
DFT of length NV is decimated (or decomposed) into successive smaller 
DFTs. One class of FFT algorithms, called radix-2 algorithms, is based on 
the assumption that N is a power of 2. The decimation is carried out in 
either the time domain or frequency domain, Accordingly, we have two 
types of algorithms in this class, viz., (a) decimation-in-time (DIT), and 
(b) decimation-in-frequency (DIF). The Cooley—Tukey algorithm belongs to 
the type (a), whereas the Sandey—Tukey algorithm to the type (b). Both the 
algorithms require VV log, N operations but differ in organization. The Cooley— 
Tukey algorithm is discussed in the next section. 


4.6.3 Cooley—Tukey Algorithm 


This algorithm assumes that ' is an integral power of 2, Le. N= 2", where m 
is an integer. The basic idea of this algorithm is to decompose the N-point 
DFT into two N/2-point DFTs, then decompose each of the N/2-point DFTs 
into two N/4-point DFTs and continuing this process until we obtain N/2 
two-point DFTs. The number of steps required to achieve this is clearly m. 
For easy understanding, we present this algorithm for N= 8 and it will be 
seen that it is easily generalized. The first step (or stage) of the algorithm is 
described below: 

Let fost, »tas---sf be asequence of values of f(t). The DFT for /, is 
given by 


j 
Fy= > feet, p= 0,1, 2,007 (4.90) 
k=0 
where 
W, =e 28. (4.89) 


We split the summation on the right side of (4.90) into two equal parts of 
length 4, one containing the even-indexed values of f(r) and the other of 
the odd-indexed values. We therefore write 


Fp = » fe Wy? + > fi W,?. (4.95) 
k(even) K(odd) 
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Putting &=2r in the first sum and & =2r+1 in the second sum of (4.95), we 
obtain 


3 Bs 
Fy => Sar we? > fare MEP, (4.96) 
r=) r=} 
But 
wePr =e 2m pr) _ gPmipr 4 _ iy pr (4.97 a) 
and 
weerr)p = Wwe = a (4.97 b) 


Using (4.97) in (4.96), we get 


3 3 
F, = fi Wy? + WP > Sars W,. (4.98) 


rei ref 


It is easily seen that the two sums on the right side of (4.98) represent 
4-point DFTs. Setting 


3 
Fi ae wp (4.99a) 
rei} 
and 
3 
F => frst We, (4.99b) 
ral 
Eq. (4.98) becomes: 
F,=FS+WPFe, pp =0,1,2,3, (4.100) 


where F, and F > are the 4-point DFTs of the even and odd-indexed sequences 
defined by (4.99). This completes the first stage of decomposing the 8-point 
DFT into two 4-point DFTs. Further, to compute (4.100) for p=4,5, 6,7 
we use the formula: 


Fy = FS 4+WpPFe 


in parc p=4,5,6,7 (4.101) 


The computations involving equations (4.100) and (4.101) for the first stage 
of the &-point DIT—-FFT are shown in the flow-graph in Fig. 4.3. 
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f 4-point DFT el \ we 


Figure 4.3 First stage of the 8-point DIT-FFT. 


In the second stage, each of the 4-point transforms in (4.100) is decomposed 


into two 2-point transforms. We then write 


3 
F, = > fr wy 
r= 


l 1 
=) fas Ww,” +WEY faser wy? 
s=0 


ral) 
_ P 
= Fg WES 


where 


l l 
i ai WP and i = > fase2 Wy. 
s=0 s=0 


Similarly, we obtain 
a oe Pp poo 


where 


1 
Fee = S fan WP and Fe =) fase. 


=O j=0 
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(4.105) 
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This completes the second stage of decomposition where each of the 
4-point transforms is broken into two 2-point transforms. The flow-graph of 
the second stage is shown in Fig 4.4. 


2-point DFT 


Figure 4.4 Second stage of the decomposition. 
From Eq. (4.102), we have 


I 
Fite) faWP = fyWy + fWP (4.106a) 
r=0 
and 
1 
Fy = > fase WY = Wy + (WP. (4.106 b) 
s=0 


Equations (4.106a) and (4.106b) show that at the third stage (which is the 
final stage, since N= 8), we obtain 


F=f Fs fo FR fe FR fe (4.107) 


It follows that, for the 8-point computation, we start with the input sequence 
fo, fa, fr, fe, fi, fs, A, and f>, and then compute the various Fourier 
coefficients. These computations can conveniently be depicted in a flow graph 
(Fig. 4.5). 
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wi wi 
fi eT 


f a. a Wa E 

r T 
Figure 4.5 Flowgraph of an 8-point DIT-FFT. 

A close inspection of Fig. 4.5 enables us to make the following observations: 


(i) The input data is shuffled and are in the order f,, fy, 44. f¢, Si. fe» a: 
and f,. They are in the bit-reversed order, as shown in Table 4.2. 


Table 4.2 Input Data in the Reversed Bits 


Input position Binary equivalent Reversed bits Index of the sequence 
0 000 000 0 
1 001 100 4 
2 010 010 2 
a 011 110 6 
4 100 001 1 
5 101 101 5 
6 110 O11 3 
7 11 111 7 


(ii) The output data for the Fourier coefficients F, is in the natural order. 


bi-e 


(iii) The computations are carried out in terms of a fundamental molecule 
called butterfly. A typical butterfly is shown in Fig. 4.6, where i 
and j represent the position numbers in the stage and m represents 
the stage of the computation. 


m me] 
' gi 1 9 
i 
' 9) gi, 
wy 1 


NM 
Figure 4.6 A typical butterfly, 


Presented By: http://www.ebooksuit.com 


SECTION 4.6: Fourier Approximation 175 


The outputs g* ' and vii are given by 
ar 


grag" +Wygh, = gf =i" -Whe" (4.108) 


where r is a variable depending on the position of the butterfly. 
The method of computation is illustrated in the following numerical 
example. 


Example 4.134 Using the Cooley-Tukey algorithm, find the DFT of the 
SeQUenice 


fh =th23.4 4,3 2, 1}. 
We have 


we =I, We = ep 2ti8 = (1-7/2, Wwe =(¢ ity = —j, 
We =-(+iif2, we=-1, we =-(1-a/,/2, 


We =i, Wy =(+iyi2 
The DIT-FFT flowgraph for DFT computation is given below in Fig. 4.7: 


20 


f=4 Oxo No \ fee F. 
birt ta I 

ee 3 Ke Yifenne, 
“SZ i” Ae ; 


F; 


Figure 4.7 Flowgraph for Example 4,14. 
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4.6.4 Sande-Tukey Algorithm 


This is an alternative approach and is a member of the class called decimation- 
in-frequency techniques. Effectively, it is the reverse of the Cooley—Tukey 
algorithm described in the previous section. However, in this case, the final 
results are scrambled. 

We start again with Eq. (4.90), viz., 


7 
Fy >> FWP p=0,1,2,...,7 
k=0 


and divide the sum in terms of the first and last four points as: 


3 7 

: pk pk 

F,=) te, +) fw. (4.109) 
k=) hood 


In the second sum on the right side of Eq. (4.109), we make a change of 
variable and write the equation as 


3 3 
F, = > Ai WPt + > fess eee, (4.110) 
k=0 k=0 
But 
plk+4) _ pp pk 4p _ (yp yy Pk 
W, =. W, =(-1) W, ; 


which is positive if p is even and negative, otherwise. 
Accordingly, (4.110) may be written as 


3 
Fa, = De + Sasa) Fy” 


k=l 


3 
=> (fe + fess) Ma, r=0,1,2,3 (4.111) 


k= 


for the even components, and 


3 
Fay) a VU = Jigsa) is 
&=0 


a 
Pee LALA 
k=0 
3 
=> fe — Sead EWE, r=0,1,2,3 (4.112) 
keQ 
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for odd components. From Eqs. (4.111) and (4.112), it can be seen that F;, 
and F5,,, are the transforms of the functions (f, + J,,4) and 
(fe - feawy, respectively. We therefore set 
fetfeea=% and (fe-feraW =u,  k=0,2,3. (4.113) 
In view of (4.113), Eqs. (4.111) and (4.112) assume the form 
F,.=5, and F,..,=f,, r=Q, 1, 2, 3. (4.114) 


It is clear that this approach can be repeated at the second stage to break 
each of the 4-point transforms into two 2-point transforms. In the general 
case, the final result is obtained after log,N stages. Figure 4.8 shows the 
flowgraph for the 8-point decimation in frequency-FFT. 


\ Lf NZX, 


f, > > 


Figure 4.8 Flowgraph for an 8-point DIF-FFT. 
Figure 4.8 shows that: 


(i) The input is in the natural order whereas the output for the frequency 
components is in the bit-reversed order. 


(ii) the computations can be carried out by using the butterfly structure. 


4.6.5 Computation of the Inverse DFT 
The inverse DFT is given by Eg. (4.91), viz. 


-kp 
Si =~» FoWy > k=0, 1, 2, ....N—-1. 
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Comparison with DFT shows that the factors wi have changed signs, the 
input and output have interchanged and that the final output is divided by 
N. Hence the flowgraph for the calculation of DFT can also be adopted for 
the computation of inverse DFT after making the above changes. 

The Fast Fourier transform is extensively employed in many areas of 
electrical engineering such as signal processing. Hence, excellent software 
packages have been developed by MATLAB, IMSL, etc. Numerical Recipes 
discusses a number of FORTRAN programs. Any of these programs can 
conveniently be used as subprograms in the solution of research problems. 


4.7 APPROXIMATION OF FUNCTIONS 


The problem of approximating a function is a central problem in numerical 
analysis due to its importance in the development of software for digital 
computers. Function evaluation through interpolation techniques over stored 
table of values has been found to be quite costlier when compared to the 
use of efficient function approximations. 

Let fi, fa. -.-y J, be the values of the given function and @), @3,...,¢,, be 
the corresponding values of the approximating function. Then the error vector 
is e, where the components of e are given by e, = /, -¢,. The approximation 
may be chosen in a number of ways. For example, we may find the 
approximation such that the quantity Jf (e. +e; +-+¢°) is minimum. This 
leads us to the least squares approximation which we have already studied. 
On the other hand, we may choose the approximation such that the maximum 
component of e is minimized. This leads us to the ‘celebrated Chebyshev 
polynomials’ which have found important application in the approximation of 
functions in digital computers. 

In this section, we shall give a brief outline of Chebyshev polynomials 
and their applications in the economization of power series.* 


4.7.1 Chebyshev Polynomials 
The Chebyshev polynomial of degree mn over the interval [~1, |] is defined by 


the relation 


T,,(x) = cos (n cos”'x), (4.115) 
from which follows immediately the relation 
T (x) =T_,(>). (4.116) 
Let cos™'x =@ so that x=cos@ and (4.115) gives 
T (x)= cos né. 


*Refer to Fox and Parker [1968] for further details and other applications of 
Chebyshev polynomials. 
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Hence 
Tp(x) =! and = 7(x)= x. 
Using the trigonometric identity 
cos (n-—1)4 4 cos (n + 1)@ = 2c0s nf cos 0, 
we obtain easily 
(x) + 7,404) = 2x7, (x), 


Tt 
which is the same as 
T4142) = 2aT,, (x) — TO). (4.117) 


This is the recurrence relation which can be used to successively compute 
all 7,(x}, since we know 7,(x) and 7,(x). The first seven Chebyshev 
polynomials are: 


To(x)=1 
T(x) =x 

T(x) = 2x" ~1 

T;(x) = 4x7 -3x (4.118) 
Ty(x) = 8x" - 8x? +1 

T; (x) = 16x° — 2027 +5x 

Tg (x) = 32x° ~ 48x4 + 18x? -1. 


The graph of the first four Chebyshev polynomials are shown in Fig. 4.9 


T lx) 


T(x) T(x) 


Figure 4.9 Chebyshev polynomials T,(x), 9=1 2, 3, 4 


It is easy to see that the coefficient of x" in T, (x) is always 2°"! Further, 
if weset y=7, (x) =cos n@, then we get 
dy 7 nsin nd 
ax sing 
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and 


dx* sin* @ 1-x* 


so that 


i] 
d-2) SZ - xB enya (4.119) 


which is the differential equation satisfied by T,,(x). 
It is also possible to express powers of x in terms of Chebyshev 
polynomials. We find 


1=Ip(x) 
— 


* = “1T(x) + T,(x)] 


1 
x? =—[37,(x)+73(x)] (4.120) 


= —(37p(x) + 474(x) + Ty(x)] 


* = = MOH (x) +57, (x)+T5(x)] 
x® ==llon(x)+ 15T>(x) +67, (x)+ Ty(*)]. 


and so on. These expressions will be useful in the economization of power series 
to be discussed later. 


An important property of T,,(x) is given by 
0, men 
a/2, m=nz0 (4.121) 


T, m=n=0 


[ Ta@Talad ae T, a os. 


-l 


that is, the polynomials 7,,(x) are orthogonal with the function 1/V(1-.x*). This 
property is easily proved since by putting x = cos @, the above integral becomes 


T,, (cos @) T,,(cos @) d@ = |} cos mé cos né dé 


rr 
| 


_[ sin(m+n)@ | sin(m—n) 6 ‘ 
| 2(m4+n) 2(m—n) |, 


from which follow the values given on the right side of (4.121). 
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We have seen above that 7, (x) is a polynomial of degree m in x and that 
the coefficient of ,* in 7, (x) is 2”"'. In approximation theory, one uses 
monic polynomials, ic. Chebyshev polynomials in which the coefficient of x” is 
unity. If P(x) 1s a monic polynomial, then we can write 


P(x)=2'"T (x), (20. (4.122) 


A remarkable property of Chebyshev polynomials is that of all monic 
polynomials, P(x), of degree n whose leading coefficient equals unity, the 
polynomial 7 ea ae a has the smallest least upper bound for its absolute 
value in the rang (1, 1). Since |7,(x)| <1, the upper bound referred to 
above is 2'“" Thus, in Chebyshev approximation, the maximum error is 
kept down to a minimum. This is often referred to as minimax principle and 
the polynomial in (4.122) is called the minimax polynomial. By this process 
we can obtain the best lower-order approximation, called the minimax 
approximation, to a given polynomial. This is illustrated in the following 
example. 


Example 4.15 Find the best lower-order approximation to the cubic 
2x3 43x". Using the relations given in (4.120), we write 


2x? +3x7 = : [7,(x) + 37;(x)] +32" 
3 1 
= 3x* +5Ti(x) +Sh@) 


=3x* +Sx4l TG) since 7) (x) =x. 


The polynomial 3x? + (3/2)x is the required lower-order approximation to the 
given cubic with a maximum error +1/2 in the range (—1, 1). 

A similar application of Chebyshev series in the economization of power 
series which is discussed next. 


4.7.2 Economization of Power Series 


To describe this process, which is essentially due to Lanczos, we consider 
the power series expansion of f(x) in the form 


f()= 4g + Axt Ax? t--t Aye", (-1sxsl) (4.123) 


Using the relations given in (4.120), we convert the above series into an 
expansion in Chebyshev polynomials. We obtain 


I(x) = By + BT (x) + B73 (x) + +--+ BT, (2). (4.124) 


For a large number of functions, an expansion as in (4.124) above, converges 
more rapidly than the power series given by (4.123). This is known as 
economization of the power series and is illustrated in Example 4.16. 
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Example 4.16 Economize the power series 
3 5 7 


+ 
6 120 5040 
Since 1/5040 = 0.000198..., the truncated series, viz., 
al 


sinx = x —-— + — i 
120 0 
will produce a change in the fourth decimal place only. We now convert the 
powers of x in (i) into Chebyshev polynomials by using the relations given 
in (4.120). This gives 


sinx » h(x)-=BR@)+BOl+ (107; (x) + 575(x) + T5(2)]. 


120 16 
Simplifying the above, we obtain 

sin x = = hG)-5 h(x) + 9 3) (ii) 
Since 1/1920 = 0.00052..., the truncated series, viz., 


sin x = 7K 1-7) (iii) 
will produce a change in the fourth decimal place only. Using the relations 


given in (4.118), the economized series is — given by 


; 169 5 § y 
sin x —— x —-—— — 3x) =—x-—x. 
192 a )e > 32 
EXERCISES 
4.1. Fit a straight line of the form Y =a) + a,x to the data: 
St ee aes 
l 2.4 4 4.2 
2 3.1 6 5.0 
ee a oe ae 
4.2, Find the values of a) and a, so that Y =a) + a,x fits the data given 
in the table: 
x ¥ 
0 1.0 
l 2.5 
2 4.8 
3 6.7 
4 8.6 
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4.3. If the straight line y =a) +a,x is the best fit to the set of points 
(X4,)4)s (.¥2)---Ons¥n) then show that 


LX, Ly, a |=0, (i=1, 2, ..., 7). 
Ex? Ey; Dx; 


4.4. Use the method of least squares to fit the straight line ¥ = a+ hx to 


the data 
x y w 
0 Z I 
] 2 | 
2 5 l 
3 11 1 
4.5, Find the values of a, 6 and c so that ¥ =a@+hx+cx" is the best fit 
to the data 
x ¥ 
0 l 
] 0) 
2 3 
3 10 
4 21 


4.6 


The table below gives the temperatures J (in 0°C) and lengths / (in 
mm) of a heated rod. If /=a,+a,T, find the values of a) and a, 
using linear least squares 


r i 

40 600.5 
50 600.6 
60 600.8 
70 600.9 
80 601.0 


4.7. Determine the normal equations if the cubic polynomial 
¥ = ag + ajx + a3x" + aye is fitted to the data (x,,y,), i=0, 1, 2, 
soap Whe 

4.8. Find best values of ap, a, and a, so that the parabola Y=dy +a,x+ax" 
fits the data: 
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x ¥ x. ¥ 
1.0 | 3.0 2.8 
1.5 1.2 3.5 3.3 
2.0 1.5 4.0 4.1 
2.5 2.6 


4.9. Determine the constants a and 6 by the least-squares method such 
that y= ae", fits the following data: 


Sa 
1.0 40.170 
12 73.196 
1.4 133.372 
1.6 243,02 


4.10. Fit a function of the form y=ar’ to the following data: 


x FF x ¥ 
2 43 20 8 
4 25 40 . 
7 18 60 3 
10 13 80 rs 


4.11, Fit an exponential function of the type y= ae”™ to the following data: 


x Bi x ¥ 
0 0.10 1.5 9.15 
0.5 0.45 2.0 40.35 
1.0 2.15 25 180.75 


4.12. The curve y=ce™ is fitted to the data: 


Secicinbeesee: eee 
1 1.5 4 40.1 
2 4.6 5 125.1 
3 13.9 6 299.5 


Find the best values of cand b. 


4.13. Fit a function of the form y= Ae* + 4,e*%* to the data given in 
the following table: 
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4.15, 


4.16. 
4.17. 
4.18. 


4.22. 
4,23. 
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od bd * ¥ 
1.0 1.175 1.5 2.129 
].1 1.336 1.6 2.376 
1.2 1.510 1.7 2.646 
1.3 1.698 1.8 2.942 
1.4 1.904 


Fit a natural cubic B-spline, s, to the data -2,-1, 0, 1, 2. Show also 
that s is unique if s(—1) is prescribed. 

Given the set of knots 0, 1,2, 3, 4, 5, 6, evaluate the B-spline of order 
6 at each of the interval knots by the divided difference method. 
Prove the Cox—de Boor recurrence formula given by (4.72). 
Repeat Problem No.15 using the Cox—de Boor recurrence formula. 
Find a Fourier series approximation to the function defined by the 
praph in Fig. 4.10. 


Figure 4.70, 


. Compute the FFT of the sequence {1, 2, 3, 4, 4, 3, 2, 1} using Sande— 


Tukey algorithm. 


. Compute the FFT of the function /(¢) defined by the points jf =1, 


f=-l =-L A=-l 4-1 6 =1 4 =L 4 =-1, using the 
Cooley—Tukey algorithm. 


. If P,(x) is any polynomial of degree n with leading coefficient 


unity, then prove that 
T,@) 
al 
Prove that x? =1/2[7)(x)+7>(x)]. 

Prove that 7,,(x) is a polynomial in x of degree n. 


<= max |P,(x). 


-lsxsl -1sx 41 
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4.24, Prove that the coefficient of x" in T,(x) is 2°"! 
4.25. Express the following as polynomials in x: 
(a) Ty(x)+ 27, (4) + He) 


(b). 2%9(x) -27(2)+ 27402) 


4.26. Express the following polynomials as sums of Chebyshev polynomials: 
(a) l+x- x +x? 
(b) 1—x? +2x". 
4.27. Obtain the best lower-degree approximation to the cubic x? + 2x7. 
4.28. Forx nearer 1, the sum 
se rr ef xt oy? 


=| Xp a ee fe ee 
art he @?o a (a0 Fao SOd0 


gives a result which is correct to five decimal places. Economize 
the above series if the fourth decimal place is not to be affected. 


4,29. Economize the series 


sinh x = gig x! 
6 120 5040° 


on the interval [-1, 1], allowing for a tolerance of 0.0005. 
4.30. Economize the series given by 


f(=l-px-2P SY, 
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Numerical Differentiation 
and Integration 


5.1 INTRODUCTION 


In Chapter 3, we were concerned with the general problem of interpolation, 
viz., given the set of values (xp, yo), (4), .¥))++--s(%_.¥,) Of x and », to find 
a polynomial @(x) of the lowest degree such that y(x) and @(x) agree at 
the set of tabulated points. In the present chapter, we shall be concerned 
with the problems of numerical differentiation and integration. That is to 
say, given the set of values of x and y, as above, we shall derive formulae 
to compute: 


2 
(i) & 2Y |. for any value of x in [xo x,], and 
de dx* 


5.2 NUMERICAL DIFFERENTIATION 


The general method for deriving the numerical differentiation formulae is to 
differentiate the interpolating polynomial. Hence, corresponding to each of 
the formulae derived in Chapter 3, we may derive a formula for the derivative. 
We illustrate the derivation with Newton's forward difference formula only, 
the method of derivation being the same with regard to the other formulae. 


187 
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Consider Newton's forward difference formula: 


y= Yp +udyy += D a2y, AeA y te GD 
where 
x = Xp +wh. (5.2) 
Then 
dy dydu } 2u-1,2 3u?-6u4+2 3 
eg ee ee hae eee fake EY ga... 5.0 
dx du de + {ay 22 PUT og wer ia 


This formula can be used for computing the value of dy/dx for non-tabular 
values of x. For tabular values of x, the formula takes a simpler form, for 
by setting x =x, we obtain w=0 from (5.2), and hence (5.3) gives 

dy 


i Oe aes oe ) 
ae) aol Apes =A" y,-—A mb 5.4) 
a le Al Yo 54 +54 Yo- aA’ yo + ( 


Differentiating (5.3) once again, we obtain 


2 7 2 
d*y = (an + = Say, pty + -| (3.5) 


from which we obtain 
Zz 
dy = (2290-4 rat +} (5.6) 
dx* ae 12 


Formulae for computing higher derivatives may be obtained by successive 
differentiation. In a similar way, different formulae can be derived by starting 
with other interpolation formuale. Thus, 


(a} Newton’s backward difference formula gives 


1 ] 1 
al +n +50 In +] vee 
and 


d’y al" ¥ Ww y lyst, 4295p +. -}, (5.8) 
ae he . eo 2 eG nt 


(b) Stirling's ‘Goal gives 


Fa _1{ Ayi+4yo 1 A’y.2+4’y1, 1 A*¥3+4*¥2 |) (5,9) 
ax xmXp h a 6 2 30 2 
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and 


l ( 2 1 wa l 6 

oe =o! A yy aA y3+—A vg 7 (5.10) 
Z 

Fale h 12 90 


If a derivative is required near the end of a table, one of the following 
formulae may be used to obtain better accuracy 


hyp =(a- fa? +a? tat Las taf eon) (5.11) 
ree ae ere ee a ee 
=| A+—A* -—A" +—A" -— AP + — AP ee |p (5.12) 
( 2 6 20 av 
i 4 Sat + Bras Ta? at — |p (5.13) 


2. 866))~—(«180—té«*~i‘C tS tCt«CS 


a laa 5s 13 6 11.7 2948 
Pi er Lee | feller Wreeatade pee Pe Ee 
2 12. 180 180 560 F114) 


vi4-yW4—-yi 4 ,i v6 wig? zi v8 + dy, (5.15) 
4 6 t 8 


-(v-iv? _lyg _ly _lgs lyse lor _lys -~ yon 
2 6 12 20 30 42 56 


=“ WP , 137 96 yy? 363 os +] 
6 180 10 560 


=(v?- fut v8 Begs tg Bot oe. (5.18) 
2. 12 180 180 560 


For more details, the reader is referred to Interpolation and Allied Tables. 
The following examples illustrate the use of the formulae stated above. 


Example 5.1 From the following table of values of x and y, obtain dy/dx 
and d"yidx? for x = 1.2: 


x J x ¥ 
1.0 2.7183 1.8 6.0496 
12 3.3201 2.0 7.3891 
1.4 4.0352 2.2 9.0250 
1.6 4.9530 


Presented By: http://www.ebooksuit.com 


190 CHAPTER 5: Numerical Differentiation and Integration 


The difference table is 


x y A a* a? a‘ a° a® 
10 27189 
0.6018 
12 3.3201 0.1333 
0.7351 0.0204 
14 4.0552 0.1627 0.0087 
0.8978 0.0361 0.0013 
16 4.9530 0.1988 0.0080 0.0001 
1.0966 0.0441 0.0014 
18 6.0496 0.2429 0.0094 
1.3395 0.0535 
20 7.3891 0.2964 
1.6359 
22 9.0250 


Here x) =1.2, Yo =3.3201 and h=0.2. Hence (5.11) gives 


EB = slo. 7351-=(0. 1627) +— =(0. 0361) -=(0. 0080) + =(0 0014) 
x=l.2 


= 3.3205. 


If we use formula (5.12), then we should use the differences diagonally 
downwards from 0.6018 and this gives 


dy} 4 4 fee 
2} 02 ale 6018 + — 50. 1333) ~(0 0294) +5 (0. 0067) = (0. 0013] 
=3,3205, as before. 


Similarly, formula (5.13) gives 


dx? 


Using formula (5.14), we obtain 


d?y I ll 5 
<2} =——|0.1627-0,0361 +—(0.0080) —=(0.0014) | = 3.318. 
| |. : Al * i! 5! 


ri 
fy = le 1333-400, 0067)+— (0 0013) = 3.32. 
"oa eee 0.04 
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Example 5.2 Calculate the first and second derivatives of the function 
tabulated in the preceding example at the point x= 2.2 and also dy/dx at 
x= 2.0. 


We use the table of differences of Example 5.1. Here x, =2.2, »,, = 9.0250 
and 4 =0.2. Hence formula (5.15) gives 


Ea mL 63594 — 7 (0.2964)+= +(0.0535) += ;(0.0094)+ 20.0014) 
dx ral “02 


= 9.0228. 
d*y l 
a2 0.2964 + +0.0535 +00. 0094) += (0 0014) | = 8,992. 
de x=22 0.04 


To find aay at x = 2.0, we can use either (5.15) or (5.16). Formula (5.15) gives 


2) ais [ 3395 + 4(0.2429) + (0.0441) + ~(0.0080) 
may 02 2 3 4 


l I 
+ Fhe all r (0.0001) 


= 7.3896. 


whereas from formula (5.16), we obtain 


aly a il Al it z= 
alte iy aa = (0.2964) = (0.0535) (0.0094) 7 (0.0014) 


= 7.3896. 


Example §.3 Find dyidx and d“y/dx* at x = 1.6 for the tabulated function of 
Example 5.1. 


Choosing xy =1.6, formula (5.9) gives 


dy wo (2a ones On36i 00H , 1 p02 + 9.0014) 
dc dvepg 0.2 2 2 2 30 2 


= 4.9530. 


Similarly, formula (5.10) yields 


d’y | I 1 
{=| =——/} 0.1988 -—(0.0080) + — (0.0001) | = 4.9525. 
de? |, 0.04 12 90 
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In the above examples, the tabulated function is e* and hence it is easy to 
see that the error is considerably more in the case of the second derivatives. 
This is due to the reason that although the tabulated function and its 
approximating polynomial would agree at the set of data points, their slopes 
at these points may vary considerably. Numerical differentiation, is, therefore, 
an unsatisfactory process and should be used only in ‘rare cases.” The next 
section will be devoted to a discussion of errors in the numerical differentiation 
formulae. 


5.2.1. Errors in Numerical Differentiation 


The numerical computation of derivatives involves two types of errors, viz. 
truncation errors and rownding errors. These are discussed below. 

The truncation error is caused by replacing the tabulated function by 
means of an interpolating polynomial. This error can usually be estimated 
by formula (3.7). As noted earlier, this formula is of theoretical interest 
only, since, in practical computations, we usually do not have any information 
about the derivative yD), However, the truncation error in any numerical! 
differentiation formula can easily be estimated in the following manner. 
Suppose that the tabulated function is such that its differences of a certain 
order are small and that the tabulated function is well approximated by the 
polynomial. (This means that the tabulated function does not have any rapidly 
varying components.) From Table 3.4 (p. 71), it is then clear that 2¢ is the 
total absolute error in the values of Ay,,4¢ in the values of A*y,, etc.., 
where ¢ is the absolute error in the values of y,. Consider now, for example, 
Stirlings formula (5.9). This can be written in the form 


4 Ay. + AY W-Y1 
A a, 7 5.19 
Ee 2 | of! abel 
where 7), the truncation error, is given by 
j 3 
Faw. | Sete ya (5.20) 
6h 2 
Similarly, formula (5.10) can be written as 
2 
x=X 
where 
Laas 
T, =—~|A" ya}. (5.22) 
Fj 1 he 2 


The rounding error, on the other hand, is inversely proportional to A in 
the case of first derivatives, inversely proportional to A’ in the case of 
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second derivatives, and so on. Thus rounding error increases as ft decreases. 
Considering again Stirling’s formula in the form. of (5.19), the rounding 
error does not exceed 2e/2h=e/h, where ¢ is the maximum error in the 
value of y;,On the other hand, the formula 


BH _ Av + Ao | Ay +A? yp, _ 
dé |i. 2h 12h 
2 ¥% ata Ye ean (5.23) 
has the maximum rounding error 
I8e _3e 
12h 2h 
Finally, the formula 
2 2 
Fg = Sy. ETM... (5.24) 
a h 


has the maximum rounding error 4e/h*, It is clear that in the case of higher 
derivatives, the rounding error increases rather rapidly. 


Example 5.4 Assuming that the function values given in the table of 
Example 5,1 are correct to the accuracy given, estimate the errors in the 
values of dy/dx and dy/dx* at x= 1.6. 


Since the values are correct to 4D, it follows that ¢ < 0.00005 =0.5x 107. 
Value of dy/dx at x = 1.6: 


3 3 
Truncation error gee Ayath . from (5.20) 
6h 2 
_ 1 0.0361 +0.0441 
6(0.2) 2 
= 0.03342 
and 
; 3¢ 
Rounding error = Sh’ from (5.23) 
_30.5)10~ 
"0.4 
= 0.00038. 


Presented By: http://www.ebooksuit.com 


194 CuapTEeR 5: Numerical Differentiation and Integration 


Hence, 


Total error = 0.03342 + 0.00038 = 0.0338. 
Using Stirling’s formula form (5.19), with the first differences, we obtain 


(+) _ Ay_y + Ay _ 0.8978 +1.0966 _ 1.9944 
de } 521 6 2h 0.4 0.4 


The exact valve is 4.9530 so that the error in the above solution is (4.9860 — 
4.9530), i.e. 0.0330, which agrees with the total error obtained above. 


Value of Py/de at x = 1.6: Using (5.24), we obtain 


a 2 
E 4 AN py, DADRE fi 
x=].6 


= 4.9860. 


de he 0.04 


so that the error = 4.9700 - 4.9530 =0.0170. 
Also, 


1 
Truncation error = a [A*y>| x 0.0080 = 0.01667 


! 
~ 120.04) 


=4 
Rounding error = 4s 4x05x10" | 0.0050. 
he 0.04 


Hence 


2 
Total error in pe = 0,0167 +0,0050 = 0.0217. 
x=1.6 


5.2.2 The Cubic Spline Method 


The cubic spline derived in Section 3.14 can conveniently be used to compute 
the first and second derivatives of a function. For a natural cubic spline, the 
recurrence formulae (3.108) or (3.109) may be used to compute the spline 
second derivatives depending upon the choice of the subdivisions. Then 
Eq. (3.106) gives the spline in the interval of interest, from which the first 
derivatives can be computed. For the first derivatives at the tabular points, 
it would, of course, be easier to use formulae (3.105) and (3.107) directly. If, 
on the other hand, end conditions involving the first derivatives are given, 
then recurrence formulae (3.111) or (3.113) may be used to compute the 
remaining first derivatives. 

The following example illustrates the use of the spline formulae in 
numerical differentiation. 
Example 5.5 We consider the function »(x)=sin x in [0,2]. 

Here M,=M,=0. Let N=2, ie.h=2/2. Then 


Yo=y,=0, y=l and My=M,=0. 
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Using formulae (3.109), we obtain 
6 
My +4M, + My =F3 —2y, + ¥) 


or 


Formula (3.106) now gives the spline in each interval. Thus, in 0< x < 7/2, 
we obtain 


which gives 
r@)=2|-Zary43] (i) 
K|l x 2 
Hence 
Z 
*(Z|-2 2 © 3) £0.71619725, 
wt 16 2| 42 


Exact value of s'(a/4)=cos a/4 =1//2 =0.70710681. The percentage 
error in the computed value of s'(7/4) is 1.28%. From (i), 


and hence 


s( 2] a2 5 ~°. = -0,60792710. 

r x 

Since the exact value is -1/./2, the percentage error in this result is 14.03%. 
We now consider values of y=sin x in intervals of 10° from x=0 to 

a. To obtain the spline second derivatives we used a computer and the 

results are given in the following table (up to x = 90°). 


y"(x) 

x (in degrees) Exact Cubic spline 
10 -0.173 648 178 0.174.089 426 
20 ~0,342 020 143 —0.342 889 233 
30 0,500 000 000 0.501 270 524 
40 0.642 787 610 -0.644 420 964 
50 0.766 044 443 -0.767 990 999 
60 0,866 025 404 0.868 226 016 
70 0.939 692 621 0.942 080 425 
80 —0.984 807 753 —O0.987 310 197 
90 —1,000.000 000 1.002 541 048 
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It is seen that there is a greater inaccuracy in the values of the spline second 
derivatives. 


5.3 MAXIMUM AND MINIMUM VALUES OF A TABULATED 
FUNCTION 


It is known that the maximum and minimum values of a function can be 
found by equating the first derivative to zero and solving for the variable. 
The same procedure can be applied to determine the maxima and minima 
of a tabulated function. 

Consider Newton’s forward difference formula 


-1 -I)(p-2 
¥=Mo + ply + PED p2 yy 4 PEP= DP =2) 435, ale vat 
Differentiating this with respect to p, we obtain 
_ 2 _ 
Day + Ba ayy 4 2E PP yg (5.25) 


For maxima or minima dwdp=0. Hence, terminating the right-hand side, 
for simplicity, after the third difference and equating it to zero, we obtain 
the quadratic for p 


eo +e,pt+epp* =0, (5.26) 
where 
1 2 1.5 
cy = Ay AY +54 Yo 
2 3 
-A*y,—A2 5.27 
c, =A" yy - Ayo (5.27) 
and 
Lys 
=A" yy. 
C2 3 ¥o 


Values of x can then be found from the relation x =x, + pA. 


Example 5.6 From the following table, find x, correct to two decimal 
places, for which y is maximum and find this value of y. 


al Ad 

1.2 0.9320 
1.3 0.9636 
1.4 0.9855 
1.5 0.9975 
1.6 0,9996 
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The table of differences is 


x y A a® 

12 0.9320 
0.0316 

1.3 0.9636 —0,0097 
0,0219 

1.4 0.9855 0.0099 
0.07120 

15 0.9975 —0,0099 
0.0021 

1.6 0.9996 


Let xg =1.2. Then formula (5.25), terminated after second differences, gives 


0 = 0.03164 a (-0.0097) 


from which we obtain p =3.8. Hence 


X = Xq + ph =1.2 + (3.8) (0.1) = 1.58. 
For this value of x, Newton's backward difference formula at x, =1.6 gives 


—0.2(-0.2 +1) 
2 


y(1.38) = 0.9996 — 0.2(0.0021) + (-0.0099) 


= 0.9996 — 0.0004 + 0.0008 
= 1.0. 


5.4 NUMERICAL INTEGRATION 


The general problem of numerical integration may be stated as follows. 
Given a set of data points (xp, yp), (x), 4) ).---+(X%_. ¥,) Of a function y = f(x), 
where f(x) is not known explicitly, it is required to compute the value of 
the definite integral 


[= [ va (5.28) 


As in the case of numerical differentiation, one replaces f(x) by an interpolating 
polynomial @(x) and obtains, on integration, an approximate value of the 
definite integral. Thus, different integration formulae can be obtained depending 
upon the type of the interpolation formula used. We derive in this section 
a general formula for numerical integration using Newton's forward difference 
formula. 

Let the interval [a, 6) be divided into » equal subintervals such that a = xy 
<u <x, <+-<x, =b. Clearly, x, =x) + nh. Hence the integral becomes 
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aoe 


my 
Approximating y by Newton's forward difference formula, we obtain 


Te 

= J | v0 + pay PE 
y 

Since x =xg + ph,de=hdp and hence the above integral becomes 


fi 
-1 -I)(p-2 
1=h {y+ pay +2B da2y, RP oe 2a yy + |p, 
i] 


p(p- D ty + p(p- ee 2) 


ayy + |e 


which gives on simplification 


i A n(2n—3) 3 n(n 2)" 3 
dv = nh — A es eee A ee (5.29) 
% : ie eo ge ee a, ee | 


From this general formula, we can obtain different integration formulae by 
putting n=1, 2, 3,...,etc, We derive here a few of these formulae but it 
should be remarked that the trapezoidal and Simpson's 1/3 rules are found 
to give sufficient accuracy for use in practical problems. 

5.4.1 Trapezoldal Rule 


Setting n=1 in the general formula (5.29), all differences higher than the first 
will become zero and we obtain 


| vacen(ro+ave)=ar+tor-m]|=t00+m 30 
= 


For the next interval [x,,x)], we deduce similarly 


J vdeetorern (5.31) 
zy 


and so on. For the last interval [x,.),x,], we have 


f vde= Bona tye) (5.32) 
*r-1 
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Combining all these expressions, we obtain the rule 


<n 
h 
[ vde=Sb0+201+¥2 tot Vat) + Yas (5.33) 


which is known as the trapezoidal rule. 

The geometrical significance of this rule is that the curve y= f(x) is 
replaced by nm straight lines joining the points (xp, yy) and (x), ¥); (4,4) 
and (x5, ¥9);...3 (X21. Mey) and (x,, ¥,_). The area bounded by the curve 
y= f(x), the ordinates x =x) and x= x,,, and the x-axis is then approximately 
equivalent to the sum of the areas of the n trapeziums obtained. 

The error of the trapezoidal formula can be obtained in the following 
way. Let y= f(x) be continuous, well-behaved, and possess continuous 
derivatives in [xo,x,]. Expanding y in a Taylor's series around x=x). we 
obtain 


x _ «ye 
| yax= { E +a) + Ee 


* 4 
We ' (5.34) 
= hyp +—Po+—— For 
vp 3 yo 6 ¥o 
Similarly, 
h* he 
=(0+ w=s Yo + Yo thoy tet wt 
ee cera (5.35) 
= hyg + 5 ¥o + r Yo + 
From (5.34) and (5.35), we obtain 


xy 


which is the error in the interval [xp, x,]. Proceeding in a similar manner we 
obtain the errors in the remaining subintervals, viz., [x,, x5 ],[¥,x;],... and 
[X,-1X,]. We thus have 

E=-L Bp + y+ ya) (5.37) 


where £ is the tofal error. Assuming that y"(¥) is the largest value of the 
m quantities on the right-hand side of (5.37), we obtain 
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Ee irs b-a ~ 
E =~-—hny"(%)=-——h y" 5.38 
mo” (x) ao? (x) (5.38) 
since nh=h-a. 


5.4.2 Simpson's 1/3-Rule 


This rule is obtained by putting m=2 in Eq. (5.29), i.e. by replacing the 
curve by n/2 arcs of second-degree polynomials or parabolas. We have then 


h 
i y de = 2h{ yo + ay +2atyg }= 200 +4 + ¥2). 
xy 
Similarly, 


h 
f aay +4y, +4) 
iy 


and finally 


h 
f 2ens (Vez + 4¥n-1 + Yn) 
Xn-2 

Summing wp, we obtain 

Ip h 
J yde=—~[% +40 + ¥y +5 tt Vn) 

xy (5.39) 

+20¥2 + Yat Ye t+ Mn-2) + nl 

which is known as Simpson's 1/3-rule, or simply Simpson's rule. [t should 
be noted that this rule requires the division of the whole range into an even 
number of subintervals of width A. 


Following the method outlined in Section 5.4.1, it can be shown that the 
error in Simpson’s rule is given by 


b 
h 
J yae=5 Do +4 On ty ts t+ Yet) 
a 
+2(¥2 + ¥4 t+ Ye tet Vn) + Yn] 


b-ai4 iyv— 
=-——A . 5.40 
130° (x) (5.40) 


where y (x) is the largest value of the fourth derivatives. 
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5.4.3 Simpson's 3/é-Rule 


Setting n=3 in (5.29) we observe that all the differences higher than the 
third will become zero and we obtain 


oh 
j yd =3h| ¥ +5Ay% +2 dy +380 | 
Xy 


3 3 I 
=3h| +501 —W)+ 702 -2) + Yo) +90 —3yy +39, -»)| 


3h 
=] Vo +3, +32 + ¥4): 
Similarly 


* 3h 
ya == Or +3¥4 + 3¥s5 + ¥6) 
%4 


and so on. Summing up all these, we obtain 


x 

i 3h 

{ y de == 100 +3 +3) + ¥3)+ (hy + 34 +35 + YQ) 
+ (Vin 3 +3) y-2 + Vn + Yn] 


=F 9 +3 +3yy +2) +3¥q +3 y5 +2 yg t-o 
+2y¥ 0-3 + 3¥ an? + 3¥ ne + ¥en) (5.41) 


This rule, called Simpson's (3/8)-rule, is not so accurate as Simpson’s rule, 
the dominant term in the error of this formula being (3/80) A’ y'"(¥). 


5.4.4 Boole’s and Weddile’s Rules 


If we wish to retain differences up to those of the fourth order, we should 
integrate between x) and x, and obtain Boole’s formula 


a4 

2h 
|» de == (yp +329) + 12y2 +3295 +Ty4) (5.42) 
<4 


The leading term in the error of this formula can be shown to be 
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If, on the other hand, we integrate between xj and x, retaining differences 
up to those of the sixth order, we obtain Weddle’s rule 


3h 
Joao Moo tintn+in+n +8490 (5.43) 
x) 
the error in which is given by —(h’/140)y"(z). 
These two formulae can also be generalized as in the previous cases. 


It should, however, be noted that the number of strips will have to be a 


multiple of four in the case of Boole’s rule and a multiple of six for Weddle’s 
rule. 


5.4.5 Use of Cubic Splines 


If s(x) is the cubic spline in the interval (x,_,,x,), then we have 


t= fyden focoae 


xy f=l -] 
na of 1 
= >= | {alls -x)M, +(x-x.4) Mj] 
fel 
2 
+2 94S mrs oboe aea{ Eu.) 


using (3.104). On carrying out the integration and simplifying, we obtain 
WE Ie 
I= yon +¥)- 7 UM, 1 Mo} (5.44) 


where Ad,, the spline second-derivatives, are calculated from the recurrence 
relation 


6 
M1 + 4M, + Mig == 1-1 — 294 + Yaad i=1, 2,..., 1-1. (3.109) 
The use of the cubic spline method is demonstrated in Example 5.12. 


5.4.6 Romberg Integration 


This method can often be used to improve the approximate results obtained 
by the finite-difference methods. Its application to the numerical evaluation 
of definite integrals, for example in the use of trapezoidal rule, can be 
described, as follows. We consider the definite integral 
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b 
I= [yde 
a 


and evaluate it by the trapezoidal rule (5.33) with two different subintervals 
of widths fh, and fA, to obtain the approximate values J, and J,, respectively. 
Then Eq. (5.38) gives the errors £, and £, as 
1 pres 
Ey, =-~(b-a)hiy"(®) (5.45) 
and 
Ey =--a)hiy'®). (5.46) 
Since the term y"(x) in (5.46) is also the largest value of y"(x), it is reasonable 


to assume that the quantities y"(¥) and p"(x) are very nearly the same. We 
therefore have 


and hence 


Since FE, - £, =J,-—/,, this gives 
2 
£, = rarie ~j). (5.47) 
We therefore obtain a new approximation /, defined by 
2 Z 
Iiim-t 
I, =J,-E, aAM BA (5.48) 
hy — hy 
which, in general, would be closer to the actual value—provided that the 


errors decrease monotonically and are of the same sign. 
If we now set 


n=sh=5h 


Eq. (5.48) can be written in the more convenient form 
{hn sh|= ai{3h)-108 (5.49) 
2 3 2 
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where J(A)=J), 
] ] 
(5a) =1 and i(m.5h}=45 


With this notation the following table can be formed 


Ith) 
, 
{h3" 14 
i[n3h Zh 

i( 5] 7G 1, phan) 
2 oe 2" 4 

(ana) 
1 1.1.1 
(Gr) (gn gh gh) 

1, 1 

ifr gh) 


The computations can be stopped when two successive values are sufficiently 
close to each other. This method, due to L.F. Richardson, is called the 
deferred approach to the limit and the systematic tabulation of this is called 
Romberg Integration. 


5.4.7 Newton-Cotes Integration Formulae 


Let the interpolation points, x,, be equally spaced, i.e. let x; =x + ih, 
i=0, 1, 2,..., 4, and let the end points of the interval of integration be 
placed such that 


Xy=a, X%,=b, h= 
Then the definite integral! 
b 
fe f y dx (5.50) 


a 


is evaluated by an integration formula of the type 


iA 
=> Cy (5.51) 
i=0 
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where the coefficients C,; are determined completely by the abscissae x,. 
Integration formulae of the type (5.51) are called Newton-Cotes closed 
integration formulae, They are ‘closed’ since the end points a and 4 are the 
extreme abscissae in the formulae. It is easily seen that the integration 
formulae derived Eqs. (5.47)+5.50) are the simplest Newton—Cotes closed 
formulae. 

On the other hand, formulae which do not employ the end points are 
called Newton—Cotes, open integration formulae. We give below the five 
simplest Newton—Cotes open integration formulae 


Hy 3 

(a) | var 2hy, + y"@), (x <¥ <x) (5.52) 
ay 
7 3h 3h? 

(b) | vee=Sor+y)+ Vr @. (Xp <¥ <x) (5,53) 
% 


E, 
(c) [> dx = Sen =). +2y5)+ hy"), (xo <¥<xq) (5.54) 


*0 


ay 
5h 95 fied 7 
(d) [ de=SF din +5 + yy +11yg)+ Thy (X), (%y <¥ < x5) 
at 
(5.55) 


a4 
(e) [ vae=Za ly, -—14y + 26y; -14yy +1 Iys)+ hy"), 


it 
(% <¥<25). (5.56) 


A convenient method for determining the coefficients in the Newton—Cotes 
formulae is the method of undetermined coefficients. This is demonstrated 
in Example 5.13. 


Example 5.7 Find, from the following table, the area bounded by the 
curve and the x-axis from x = 7.47 tox = 7.52 


x F(x) - F(x) 
7.47 1.93 7.50 2.01 
7.48 1.95 7.51 2.03 
7.49 1.98 7.52 2.06 
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We know that 
7.52 
Area = | F(x)dk 
7A7 
with h=0.01, the trapezoidal rule (5.32) gives 
0.01 


Area = =u 93 + 2(1.95 +1.98 + 2.01 + 2.03) + 2.06] = 0.0996. 


Example 5.8 <A solid of revolution is formed by rotating about the x-axis 
the area between the x-axis, the lines x =0 and x =1, and a curve through the 
points with the following coordinates: 


x ¥ 
0.00 1.0000 
0.25 0.9896 
0.50 0.9589 
0.75 0.9089 
1.00 0.8415 
Estimate the volume of the solid formed, giving the answer to three decimal 
laces. 
: If F is the volume of the solid formed, then we know that 
I 
Van | vax 
0 


Hence we need the values of y° and these are tabulated below, correct to 
four decimal places 


* # 
0.00 1.0000 
0.25 0.9793 
0.50 0.9195 
0.75 0.8261 
1.00 0.7081 


With A=0,25, Simpson's rule gives 
V= =—"1.0000 +4(0.9793 + 0.8261) + 2(0.9195) + 0.7081] 


= 2.8192. 
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Example 5.9 Evaluate 


correct to three decimal places. 

We solve this example by both the trapezoidal and Simpson’s rules with 
h=0.5,0.25 and 0.125 respectively. 

(i) A=0.5: The values of x and y are tabulated below: 


x ¥ 
0.0 1.0000 
0.5 0.6667 
1.0 0.5000 


(a) Trapezoidal rule gives 

ia [1.0000 + 2((0.6667) + 0.5] = 0.70835. 
(b) Simpson's rule gives 

[= 5 [1.0000 + 40.6667) + 0.5] = 0.6945. 


(ii) #A=0.25: The tabulated values of x and y are given below: 


* J 
0.00 1.000) 
0.25 0.8000 
0.50 0.6667 
0.75 0.5714 
1.00 0.5000 


(a) Trapezoidal rule gives 
f= sl .0+ 2(0.8000 + 0.6667 + 0.5714) + 0.5] = 0.6970. 
(b) Simpson's rule gives 


fz mil 0 +.4(0.8000 + 0.5714) + 20.6667) + 0.5] = 0.6932. 
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(iii) Finally, we take 4 = 0.125: The tabulated values of x and y are 


x Bg x hy 
0 1.0 0.625 0.6154 
0.125 0.8889 0.750 0.5714 
0.250 0.8000 0.875 0.5333 
0.375 0.7273 1.0 0.5 
0.5 0.6667 ~ 


(a) Trapezoidal rule gives 


re 2 {1.0 + 2(0.8889 + 0.8000 + 0.7273 + 0.6667) 
+ 0,6154 +.0,5714+0,5333) + 0.5] 
= 0.6941. 
{b) Simpson's rule gives 
T= 51.0 + 4(0.8889 + 0.7273 + 0.6154 + 0.5333) 


+ 2(0.8000 + 0.6667 + 0.5714) + 0.5] 
= 0.6932, 


Hence the value of J may be taken to be equal to 0.693, correct to three decimal 


places. The exact value of J is log,2, which is equal to 0.693147.,.. This 


example demonstrates that, in general, Simpson’s rule yields more accurate 


results than the trapezoidal rule. 
Example 5.10 Use Romberg’s method to compute 
| 
1= [{——a, 
1+x 
0 


correct to three decimal places. 


We take A = 0.5, 0.25 and 0.125 successively and use the results obtained in 


the previous example. We therefore have 
I(h) = 0.7084, (54) =0.6970, and i( +a) = 0.6941 
Hence, using (5.49), we obtain 


rf ( in) = 0.6970 + +(0.6970 — 0.7084) = 0.6932. 
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1( 5h | = 0.6941 + 5 (0.6941 ~0,6970) = 0.6931 


Finally, 
i(nghth) = 0,6931+ ; (0.693 1 — 0,6932) = 0.6931. 


The table of values is therefore 


0.7084 
0.6932 

0.6970 0.6931 
0.6931 

0.6941 


An obvious advantage of this method is that the accuracy of the computed 
value is known at each step. 


Example §.1]7| Apply trapezoidal and Simpson's rules to the integral 
| 
f= J 1-x2dx 


0 
continually halving the interval A for better accuracy. 


Using 10, 20, 30, 40 and 50 subintervals successively, an electronic computer, 
with a nine decimal precision, produced the results given in Table below. 
The true value of the integral is w/4=0.785 398 163. 


No. of subiniarvals Trapezoidal rule Simpson's's rule 


10 0.776 129 582 0.781 752 040 
20 0.782 116 220 0.784 111 766 
30 0.783 610 789 0.784 698 434 
40 0.784 236 934 0.784 943 836 
50 0.784 567 128 0.785 073 144 


Example §.12 Evaluate 
i 
I= [si nx dx 
0 


using the cubic spline method. 


The exact value of J is 2/m=0.63661978. To make the calculations easier, 
we take n=2, i.e. h=0.5, In this case, the table of values of x and 


y=sinax is 
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x ¥ 
0 0 
0.5 1.0 
1.0 0.0 


Using (3.109) with A4, =, =0, we obtain Mf, =-12. Then formula (5.44) 
gives 

] | 1 1 
[=—(yp + Wm Tay le + +1 +92) +My) 


4 92 
1 1 1.4 
=—+—+—-4+— 
4 16 4 16 
_5 
8 
= 0.62500000; 


which shows that the absolute error in the natural spline solution is 0.01161978. 
It is easily verified that the Simpson’s rule gives a value with an absolute 
error 0.03004689, which is more than the error in the spline solution. 


Example 5.13 Derive Simpson’s 1/3-rule using the method of undetermined 
coefficients. 
We assume the formula 


h 
J ver=ayyy+agy tay, (i) 
—h 
where the coefficients a_,,a) and a, have to be determined. For this, we 
assume that formula (i) is exact when p(x) is 1, x or x, Putting therefore 
y(x)=1,x and x* successively in (i), we obtain the relations 


A 

@_1 +0) +4, = | ae=2%, (ii) 
-h 
h 

-ay+a= | xdx=0 (iii) 
_h 

2 | 
and a +a =Fh. (iv) 


Solving (ii), (iii) and (iv) for a_,, ag and a,, we obtain 


a,=2= and _ 4h 
1-3 aq % 3" 
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Hence formula (i) takes the form 


A 
h 
f yde=s(y-1 +4¥q + v1), 
=k 
which is the Simpson's 1/3-rule given in Section 5.4.2. 


5.5 EULER-MACLAURIN FORMULA 


Consider the expansion of lMe*-1) in ascending powers of x, obtained by 
writing the Maclaurin expansion of e* and simplifying 


ze ee ee er ere (5.57) 
e-l x 2 
where 
By, =0, B= B= B= 4, ete. 
. 12° 720, 30,240 


In (5.57), if we setx=ADand use the relation E =, {see Section 
3.3.4), we obtain the identity 


eee ee B,hD + Bh’ D? + Bgh® D? +--- (5.58) 


or equivalently 


Sia A _i nh n 3 nm . (5.58) 
Sat wp ee ee D?(E"-1)+ 


Operating this identity on yg, we obtain 


E" -1 1 


1 on 
Fl “Rye ~ Dm - 5 —l¥p + BADE" —l)yq +--: 


1 1 a Ul iF wr 
= 5 ln — Yo) Fn ~ Yo) + BihOn ~ Yo) + Bah? (yn 90") 


* Bsh°(y," = Jt (5.59) 


It can be easily shown that the left-hand side denotes the sum yp + + ¥2 +°* 
+y,-;, Whereas. the term 


1 
kb” — Yo) 
on the right side can be written as 
if 
rf ede 
“g 


since 1/D can be interpreted as an integration operator. 
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Hence, Eq. (5.59) becomes 


Zz 
fy de = 590 +2y t2yQ tet 2y_ 4 + SOs = Yo) 
Ey 
Pe 48 
790" yo )- arr ae Yo) (5.60) 


which is called the Euler—Maclaurin’s formula for integration. The first 
expression on the right-hand side of (5.60) denotes the approximate value 
of the integral obtained by using trapezoidal rule and the other expressions 
represent the successive corrections to this value. It should be noted that 
this formula may also be used to find the sum of a series of the form 
Yo + Yi + ¥2 +++ y,. The use of this formula is illustrated by the following 
examples. 
Example 5.14 Evaluate 

ai2 

r= [ sinxdx 

0 

using the Euler—-Maclaurin's formula. 


In this case, formula (5.60) simplifies to 


mii A? a 1° 
=—(yp +2y, +2y5 +---+2y,_, +9, 4+— ———++. fi) 
J sin x dx = * (90 v1 ¥2 ¥n-t Yn) Ti 70° 30.240 


To evaluate the integral, we take h=s/4. Then we obtain 
nid 


a 
dx = = (04240) 4+ 
pas 7O+2+0) +75 + ot 
2 n* 
i Tt 5 
== +--+ See tel 
4 192184320 
= 0.785398 +0.051404 +0.000528 
= 0.837330, 
On its Giker hari Sai Wate ae Se 


ni2 


| sin xdx = a[o+ 2(0.382683) +.707117 + 0.923879 + 1.000000) 
0 


= 0.987119 + 0.012851 + 0.000033 
= 1.000003. 
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Example 5.15 Use the Euler—Maclaurin formula to prove 


2 _a(n+12n+1) 
pee 


In this case, rewrite Eq. (5.60) as 


‘ee he h ne 
= + + fp a oh - i = ay +— # na f ee vr ree 
Mott y2 Yn +5 Yn nd? ried yo) 739 Yo ) 
+o yt - yf) ( 
30,2407 " 7" 
Here p(x) =x", y'(x) = 2x and A=]. 
Hence eq. (i) gives 


I ! 
Sum = ig de+ ("7 +1)+5@n-2) 
~ lop -n+ tor + n+ -1) 
3 2 6 


= £m? +3n? +n) 


_ n(n + I(2n +1) 
= 


5.6 ADAPTIVE QUADRATURE METHODS 


We have so far considered integration formulae which use equally spaced 
abscissae. In practical problems, however, we often come across situations 
which require the use of different step-sizes while solving a problem. This 
would be so if the interval in question contains parts over which the function 
varies too rapidly or too slowly. For better accuracy and efficiency, it would 
be desirable to take a smaller size in parts of the interval over which the 
function variation is large. Similarly, it would be efficient to take larger step 
sizes over parts in which the function varies too slowly. A numerical integration 
procedure which adopts automatically a suitable step-size to solve an integration 
problem numerically is called adaptive quadrature method. We describe 
below an ‘adaptive quadrature method’ based on Simpson's (1/3)-rule and 
this can easily be modified to the other integration formulae: 
Suppose that we wish to approximate the integral 


a] 
I= | y(x)de (5.61) 
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to within an accuracy «> 0. Using Simpson's (1/3)-rule with A =(b-a)/2, we 
obtain 


I= freracat A] a) +4y2%bs yoy] 2 y® Gi), a<g <b 


(b—ayh* iy 
= I(a,b)-— y"(G), (5.62) 


where 


(a, 6) =— AI y(a)s aye? + v()| (5.63) 


Now, we subdivide the interval and set h =(b-a)/4. Simpson’s (1/3)-rule then 
gives 


eo 2y AZO + Ay a+35 


b 
fu frayae= 4 yay ay + | 
ale h'(6-a) 2 y¥ 


TP (62) 


+ 


3a+b eet), 
4° 2 


= Heaes 


h| até aS (b- a)n* = 
+ [ys 3 +4 +0)|- 180 x sae oe 


- (ats!) o- y"(), (5.64) 


where 


1222) =| yas 4y “are ya) (5.65a) 


| ee y2i® + | (5.65) 
Assuming 


y"(&)=y"(E) 
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Eqs. (5.62) and (5.64) give on simplification 


ae 7 a+b) {a+b = a iv 


Substituting (5.66) in (5.64), we obtain an estimate for the error, viz. 


b 
a+b a+b 
J x@ae- i(a ; arb). (22? 5 
l at+h a+b 
=— - — {| —— 5.67 
5 |/(a.b) i(a, ; i r +) (3.67) 
If we suppose 
l atb a+b 
mi o)- i(a, 3 )-{ 3 oJ <e (5.68) 


for some ¢>0 in the interval [a, 5], then Eq. (5.67) means that 


Je -i{ ox?) (22? 6) ce (5.69) 
and that 

; b b 

f xepacas{ a2 rice 6} (5.70) 


to within an accuracy of ¢> 0. 

If the inequality (5.68) is not satisfied, then the procedure is applied to 
each of the intervals [a,(a+5)/2] and [(a+6)/2,5] with the tolerance 
ef2. Ifthe inequality is satisfied in both the intervals, then the sum of the 
two approximations will give an approximation to the given integral. If the 
test fails in any of the intervals, then that particular interval is subdivided 
in to ‘two subintervals’ and the above procedure is applied with a tolerance 
which is half of the previous tolerance. The following example demonstrates 
the testing procedure. 


Example 5.16 Test the error estimate given by (5.67) in the evaluation of the 
integral 
ai2 
[= [ c0s x de. 
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Let h= 7/4. Then 


(0.5) E( 147 +0)=1.00208 


2) 12\  f2 
Also 
a) l 
.— |=—] 1+4c0s—+— 
i(0,2) (1 eo F) 
and 
a 3 x{ i ci 
[| —,— |=—| + 4cos— +0). 
(; 4 £{ ts cor +0) 
Hence 


It follows that 
1 


il 5)-(04) (3-5) 


It can be verified that the 
ni2 


[eosxae-1(0.7)}-1(7-5] 


which is less than that obtained above. 


(0.0021 5) = 0.00014. 


Actual error = 


5.7 GAUSSIAN INTEGRATION 


We consider the numerical evaluation of the integral 


b 
im | 4@ de. 


In the preceding sections, we derived some integration formulae which 
require values of the function at equally-spaced points of the interval. Gauss 
derived a formula which uses the same number of function values but with 


different spacing and gives better accuracy. 
Gauss’ formula is expressed in the form 


| A 
| F(u) du = W,F (4) + WF (uy) +--+ WF (u,)= >. WFC), (5.72) 


where the HW’, and w; are called the weights and abscissae, respectively. An 
advantage of this formula is that the ‘abscissae and weights’ are symmetrical 


with respect to the middle point of the interval. 


Presented By: http://www.ebooksuit.com 


Hidden page 


Presented By: http://www.ebooksuit.com 


Hidden page 


Presented By: http://www.ebooksuit.com 


Secnion 5.7: Gaussian Integration 219 


As an example, when n=] we solve 7,(u)=0, |e. 


Lear 9 
— (3u" -1)=0, 
5 f ) 
which gives the mo abscissae: 
3 
aan aes 2 and meen ae C4 
4. 63 j3 3 


Similarly, for n=3 we solve P,(w)=0, That is, 

= (G58 —30u* +3) =0, 
which gives the four abscissae: 

V2 
er 15+ 2,/30 . 
35 

The weights ’, can then be found from (5.82). It should be noted, however, 
that the abscissae w, and the weights W, are extensively tabulated for different 


values of nm. We list below, in Table 5.1, the abscissae and weights for 
values of nm up to n=6., 


Table 5.1 Abscissae and Weights for Gaussian Integration 


n $uj Wi 
0.57735 02692 1.0 
0.0 0.88888 86888 
0.77459 66692 0.55555 55556 
4 0.33998 10436 0.65214 51549 
0.86113 63116 0.34785 48451 
5 0.0 0.56396 88889 
0.53846 93101 0.47862 86705 
0.90617 98459 0.23602 68851 
6 0.23861 91861 0.46791 39346 
0.66120 93865 0.36076 15730 
0.93246 95142 0.17132 44924 
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which is singular at f= x. The principal value, P(/), of the integral is defined 
by 


P(I)= in| | J< LO a+ J Aa (a<t<b) 
c= 
a (5.85) 
=/(}), for f<a or f>6. 


Setting x=a+uh and t=a+hkh in (5.84), we obtain 
p 
P(I) =P fee a. 
; u-k 


Replacing f(a+ah) by Newton's forward difference formula. Table 3.1 (p.66) 
(see Section 3.6) at x=a and simplifying, we have 


n=y ££O,,, (5,86) 
jo 
where the constants c, are given by 
F . 
cj =P {hu (5.87) 
7 5 un-k 


In (5.87), (wg =1, (uw), =u, (uw), =u(u—1), ete. Various approximate formulae 
can be obtained by truncating the series on the right side of (5.86). Thus, by 
writing (5.86) in the form 


i 
1,(1)= yah Le)... (5.88) 
joo 


we obtain rules of orders 1, 2, 3,... etc., by setting n=1,2,3,... respectively. 
(a) Two-point rule, m= 1: 


1 Ad 
hif= ye, 


j= 
= ¢of(a)+ cA f(a) (3.89) 


= (cg — ce) f(a) +e f (ath). 
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(b) Three-point rule, n=2: 


2 af 
b(f)= yee, 


f=0 


=cof(a)+eA f(a)+e,A7 f(a) 


(40-4 hen fla) en) flath)+ crf a + 2h). (5.90) 


In the above relations, the values of c, are given by 


cj = ptegk (5.91) 
2 
C3 = 5 + lk =I) + cok(k~D. 


A discussion of errors in these formulae may be found in the paper by 
Delves [1968]. 


5.8.2 Generalized Quadrature 


In evaluating singular integrals which arise in practical applications, it will 
often be convenient to develop special integration formulae. 

We consider, for instance, the numerical quadrature of integrals of the 
form 


b 
K(s)= | FO 9C-s)ar, (5.92) 


where f(t) is continuous but @(u) may have an integrable singularity, e.g. 
log|s—*| or |s—r|" for a> -1. For the numerical integration, we divide 
the range (a, 5) such that ¢,; =a+ jh(j=0, 1, 2,..., 9), with b=a+ah, 
Then (5,92) can be written as 


n=} ‘j+1 
Ksy= yf fe ge-syat. (5.93) 


geo ny 


The method to be followed here is to approximate f(r) in (5.93) by the 
linear interpolating function /,(¢), where 


full) = [byes - 4G) +04) F Gaadh (5.94) 
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Substituting f.(f) for f(t) in(5.93), we obtain 


n=) fj+ 
Ks) -— YJ te-9 £6) 40-1) £6) OO- sat 
j=0 ry 
Setting f=; + ph, this becomes 


n=] 


I 
Ks)=h Sf (= p)fee))+ pf lejadlo t+ ph—s)dp, 


fed o 
which can be written as 


Hs=h ¥ la fltj)+ Bf jadh (5.95) 
jo 
where 
1 
aj=h| (-p)o(t; + ph—-s) dp (5.96a) 
0 
and 
I 
B,=h{ pot; + ph-s) dp. (5.96b) 
0 


It is clear from (5.96a) and (5.96b) that if @(u) =1, then a, = B j= Ah/2, and 
hence Eg. (5.95) gives 


(3) = SUF ig) + 261 )+ 2F ) +--+ 2f ya) + qh 


which is the trapezoidal rule deduced in Section 5.4.1. Hence the rule defined 
by (5.95), (5.96a) and (5.96b) is called the generalized trapezoidal rule and 
is due to Atkinson [1967]. When @(u)=log|u|, this rule finds important 
applications in the numerical solution of certain singular integral equations.' In 
practice, the computation of the weights a, and 8, may be difficult, but 
they can be evaluated once and for all, for a given @(w). 

In a similar way, one can deduce the generalized Simpson's rule— 
analogous to the ordinary Simpson's rule—by approximating f(r) by means 
of a quadratic in the interval (4), ¢,))-" 

The error in generalized quadrature can also be estimated by the method 
outlined in Section 5.4.1. For example, it can be shown that the error in the 
generalized trapezoidal rule is of order h*, assuming that f” is continuous 
in [a, b). 


lL. See, for example, Sastry (1973, 1976]. 
2. See, Noble [1964], p. 241. 
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5.9 NUMERICAL CALCULATION OF FOURIER INTEGRALS 


We consider, in this section, the problem of computing definite integrals 
which involve oscillatory functions, i.c. integrals of the form 


b 
I= [ £0) 0s ax dx (5.97) 
a 


b 
1,= | £@)sin ox dx, (5.98) 


Such integrals, called the Fourier integrals, occur in practical applications, 
e.g. spectral analysis. We describe below three methods for the numerical 
integration of such integrals and compare their accuracies through a numerical 
example. Only the outlines of the methods will be indicated here. For details, 
the reader is referred to the research papers cited. 
For definiteness, we outline the methods with reference to the particular example 
T= i e cos wx dx = 
rf l+o@ 
but these methods also hold good for equations of the type (5.98). In all the 
formulae below, a step-length A is used. 


5.9.1 Trapezoidal Rule 
Using this rule, the integral in (5.99) approximates to 


(5.99) 


a 


‘ ] gi-lHa)h 
eer 


1 cos wh +i sin ah 
=h|—+Re ———————— 
2 cosh h+sinh h—cos mh—isin ah 
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which gives on simplification 


A sinh (5.100) 
2 cosh A—cos wh 


Formula (5.100) is due to Einarsson [1972]. 


i 


5.9.2 Filon’s Formula 


In his original paper, Filon [1928] derived formulae for integrals of the type 
(5.97) and (5.98). In his method, the interval [a, 5] is divided into 2N 
subintervals and in each dowble interval, f(x) is approximated by a quadratic. 
Thus this rule is similar to Simpson's rule except that there is an extra 
factor cos wx in the present case. Since the derivation of the formula is 
quite involved, only the relevant details are given below. 

With A=(b-a)(2N), let 


C -5 f(a) cos ma + f (a+ 2h) cos w(a+ 2h) 


+ f (a+ 4h) C08 (a+ 4h) +45 f(b) 008 wb (5.101) 


C, = f(a+h) cos w(a+h)+ f (a+ 3h) cos w(a+3h)+-- 
+ f(b—h) cos w(b —h). (5.102) 
Then, 


b 
J i (x)cos wx dy =A{aLf(b) sin ab — f(a) sin wa] + fC) +dC,}, (5.103) 
where 


ah? + @h sin oh cos wh-2 sin? (anit) 


g= 33 


@ 
_ 2 {ah [1 + cos” (wh)] - sin 2a} (5.104) 
wo 
_ 4(sin wh — ah cos wh) 
eer Fue 


A similar formula for the integral (5.98) is given by 


B 


é 


i 
i f(x) sin wx de = h{-a [f(b) cos wb — f(a) cos wa] + BS, +655}, (5.105) 
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where 5, and §, are sums similar to C, and C, for f(x) sin wx. 
For the integral in (5.99), the Filon formula is given by 


Tel, =H p( de cos 20h +e co daoh+-~ 


+ g(e eos ohsecos30h +-)} (5.106) 
where 
1+cos" (wh) _ sin 2h 
=? (5.107a) 
oh? ok ae 
and. 
sin@h cos @h 
ve (oe on ar) eae 


For computational purposes, however, the right side of (5.106) can be put 
into a more convenient form (see Einarsson [1972]). We have 


ss] 


“24 eos Jah +e" cos dah ++ => + e 2" cos Jwnh 


om 
+Re De e72nh eiionh 


asl 


1 
—+2e 
Z 


gi ltiw)inh 


_l sinh 2h 
2 cosh 2h-cos 2wh- 
on simplification. 
In a similar manner, we obtain 


sinh h—cos ah 


-h -3h 
e cos@h+e ~ cos Jah +---=—————_—__. 
cosh 24 —cos 2a@A 
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Hence (5.106) becomes 
Eaate sinh 2h 7 sinh A.cos ah 
2 "(2 cosh 2h—cos 2wh 7 cosh 2h— cos 2h 


Asinh h 


—————— 5.10 
mare hamoaa cosh 4+ cos at), ( 8) 


where p and g are given by (5,107). 


5.9.3 The Cubic Spline Method 


We first consider the integral (5.97) and let the interval [a, 5] be divided into 
n equal subintervals, each of which is of length h=(b-a)/n. Let f(x) =),, 
i=0, l,...,. [fs(x) is the cubic spline interpolating to the data values (x,, y,), 
then we have 


3 3 
s00)= Mp D4, EH” fy, ~ Mist yp |= 


6h h 
+» ~My?) Fa, (5.109) 


where the M,[=s"(x,)] satisfy the recurrence relation 
6 
M,., + 4M, + Mis) 52 ret —2y; + Yih i=1,2,....m-1. (5.110) 


Hence we have 


& b 
i= [ £02) 0s wx de = [ 502) 00s cox ay. 
a ci 


Since s(x) €e[a, 5], we can integrate the above integral three times and 
obtain 


b & : 
L. =| 92 = [eo ae 
® Je ’ a 


= ~{s(b)sinbe — s(a)sin aw] + feos be s'(b)-—cos aw s'(a)) 
ra 


b 
-+{s"(b) sin ba — s"(a)sin aa) + + sin wx s°'(x) de. (5.111) 
@ @ 


a 
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Similarly, applying Simpson's rule to the integral 


Vj+l Mal 
fx | f(x, ya ay, (5.118) 
¥y-1 4-1 
we obtain 
Vy<t 
A 
1-5 (F144 FG Y)+ Seay Mey 
¥ j= 


~ x (XY pad t AF) + FOV jo) 


+ 4CF (2). V4) t 4 fp y+ SOY pa} 


+ I (Xia »¥j-1) + Af (X44 Fj) + F(X jay oP p+ 


hk 
= 9 bist + Fienyat + Foye + Fist, ys 
+ 4Cfiay + Spat Aja t Fp) + 164, /)- (5.119) 


A numerical example is given below. 
Example 5.18 Evaluate 


i | 
f= [ eaeay, 
o 6 


using the trapezoidal and Simpson's rules. With = =0.5, we have the 
following table of values of e**”. 


ee a ee nara 
¥ 0 0.5 1.0 
0 l 1.6487 2.7183 
0.5 1.6487 2.7183 4.4817 


1.0 2.7183 4.4817 73891 
Using the ‘trapezoidal rule’ (5.117) repeatedly, we obtain 


i= 


= (1.0 + 4(1.6487) + 6(2.7183) + 4(4.4817) + 7.3891] 


_ 12.3050 
4 
= 3.0762. 
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Using ‘Simpson’s rule’ (5.119) repeatedly, we obtain 


I = 92 [1.0+2.7183 + 7.38914 2.7183 


+ 4(1.6487 + 4.4817 + 4.4817 + 1.6487) + 16(2.7183)] 


_ 26.59042 
9 


= 2.9545. 


The ‘exact value of the double integral is 2.9525’ and therefore it can 
be verified that the result given by Simpson's rule is about sixty times more 
accurate than that given by the trapezoidal rule. 


EXERCISES 


5.1. Find £(Jp) at x=0.1 from the following table: 


a4 Jg(x) 
0.0 1.0000 
0.1 0.9975 
0.2 0.9900 
0.3 0.9776 
0.4 0.9604 


5.2. The following table gives the angular displacements @ (radians at 
different intervals of time f (seconds) 
é t e t 
0.052 0 0.327 §=©0.08 
0.105 0.02 0.408 0.10 
0.168 0.04 0489 0.12 
0.242 0.06 


Calculate the angular velocity at the instant t= 0.06. 
5.3. From the following values of x and y, find dy/de when x=6: 


* ¥ * ¥ 
45 9.69 6.5 26.37 
5.0 12.90 7.0 32.34 
5.9 = «16.71 7.3 39.15 
6.0 21.18 
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5.4. .A rod is rotating in a plane. The following table gives the angle @ 
(radians) through which the rod has turned for various values of 
the time ¢ in seconds. Find the angular velocity of the rod when 


t= 0.6, 
x ¥ x ¥ 
0 0 08 2.022 
0.2 O.122 1.0 3.200 
0.4 0,493 12 4,666 
0.6 1.123 


5.5. The following table of values of x and y is given: 


x y x y 
0 6.9897 4 8.4510 
l 7.4036 4 8.7506 
2 7.7815 6 9.0309 
3 8.1291 
Find dyed when (i) x =1, (ii) x =3, and (iii) x =6. Also find d* p/dx* 
when x=3. 
5.6. A function y= f(x) 1s defined as follows: 
x y= f(x) x y= f(x) 
“1.0 1.0 1.20 1.095 
1.05 1.025 1.25 1.118 
1.10 1.049 1.30 1.140 
1.15 1.072 


Compute the values of dy/dx and dyide* at x =1.05. 


§.7. Tabulate the function f(x)=5x*-3x°+10x-6 at xp =—0.50, x, =1.00 
and x, = 2.00. Compute its first and second derivatives as accurately 
as possible. Compare your results with the true values. 


5.8. The distances travelled by a rocket at different times are as given 


below: 
f 5 f 5 
0 0 4 38 
| 3 5 50 
2 7 
3 15 


Estimate the rocket’s velocity and acceleration for each value of fF. 
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5.9. 


5.10. 


S.11. 


§.12. 


A cubic function y= f(x) satisfies the following data: 


x F(x) 
0 l 
! 4 
3 40 
4 85 


Determine the function f(x) and hence find '(2) and f"(2). 

The temperature T of a cooling body drops at a rate which is 
proportional to the difference 7-7, where 7, is the constant 
temperature of the surrounding medium. A metal ball which is initially 
at 150°C is dropped into water that is held at constant temperature 
T, = 40°C. The temperature of the ball at time r is given as follows: 


r(in min.) T (in °C) 


0 150 

5 74.8 
10 68.5 
15 50.7 
20 44,4 


Determine d7/dr at each value of ¢. If dT/dt =-k(T -T,), estimate 
the value of & by linear least squares. 


The function y=3xe™* is tabulated below: 


x ¥ 

3 0.448 1 
3.2 0.3913 
5 0.1010 


Develop a subprogram to find the first derivative values of y, test 

it with the above data and compare your results with the actual 

values. 

From the following table of values of x and », find dy/dx at x =2 
using the cubic spline method. 


¥ 
2 11 
3 49 
4 123 
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5.14, 


5.15. 


5.16. 


5.18. 
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From the following table of values of x and », determine the value 
of dy/dx at each of the points by fitting a cubic spline through them. 


x y 
1 I 
2 3 
d 4 
5 2 


If y=A+ Bx + Cx? and ¥o. ¥is J) are the values of y corresponding 
to x=a, a+h, a+2h respectively, prove that 
a+ih 


i 
J ydr=> (Yo +4y, +45). 


Evaluate 


ft i 
(a) [rsinedt o) [a 
4 5+ 2 
=% 
using the trapezoidal rule. 
Discuss a method for finding an approximate area under a given 


curve. A curve is given by the points of the table given below: 


x ¥ * ¥ 
0 23 2.3 16 
0.5 19 3.0 19 
1.0 14 3.5 20 
la ll 4.0 2) 
2.0 12.5 


Estimate the area bounded by the curve, the x-axis and the extreme 
ordinates, 


. Estimate the value of the integral 


: I 
Joa 
1 
by Simpson’s rule, with 4 strips and 8 strips, respectively. Determine 


the error by direct integration. 


Evaluate 
wid 


| Jsin@ dé, 
i] 


Using Simpson's rule with f=7/12. 
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3.19. Find the value of 


5.20. 


§.21. 


Sade 


[ tiogrds 
3 


by taking 4 strips. 
The velocities of a car (running on a straight road) at intervals of 
2 minutes are given below. 


Time (in min.) Velocity (in km/hr) 

0 0 
2 22 
4 30 
6 27 
g 18 
10 7 
12 


Apply Simpson’s rule to find the distance covered by the car. 
Compute the values of 


ae 
0 


l+x? 


by using the trapezoidal rule with #=0.5, 0.25 and 0.125. Then 
obtain a better estimate by using Romberg’s method. Compare your 
result with the true value. 


A reservoir discharging through sluices at a depth A below the water 
surface has a surface area A for various values of / as given below, 


A (in ft) A (in sq.ft) 
10 950 
11 1070 
12 1200 
13 1350 
14 1530 


If ¢ denotes the time in minutes, the rate of fall of the surface Is 
given by 

dh 98 7 

dit A 
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5.23. 


5.24, 


5.25, 


5.26. 


5.27. 


5.28. 


Estimate the time taken for the water level to fall from 14 ft to 10 ft 
above the sluices. 


Find the approximate value of 
ara 


[ Yeos@ ae 


0 
by dividing the interval into six parts. 


Evaluate 
1 
| cos x dx 
it] 
using #=0.2. 
Determine the maximum error in evaluating the integral 
wii 
| cos x dx 


by both the trapezoidal and Simpson’s rules using four sub-intervals. 
Estimate the value of 


* sint 

| 
[a 
q I 


Derive Simpson's (3/8)-rule 
| yd == h(yy +39 + 3yy + 5). 


=p 
Using this rule, evaluate 


with A=1/6. Evaluate the integral by using Simpson’s (1/3)-rule 
and compare the results. 
Deduce Weddle’s rule 
3h 
y de =a 00 +5 + Hy + O95 + Vg +55 +5) 
% 


and use it to obtain an approximate value of 7 from the formula 


| 
iT | 
aes be 
4 li 
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5.36. 


5.37. 


5.38. 


3.29, 


3.40. 


5.41. 
5.42. 


5.43. 
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Use the three-point Gauss-Legendre formula to evaluate the integral 
ara 
Sin x ax. 
a 
Compare this result with that obtained by Simpson’s rule using 
seven points. 
Use the method of undetermined coefficients to derive the formula 
if 
| f(x) sin x de = f(0) — f(20). 
a 
Apply Filon’s formula to obtain the value of 
at 
j log (1+) sin 10w dw. 
1] 


Using Filon’s method, evaluate the integral 
lr 
| e* sin 10¢ df. 
0 
Compare your result with the analytical solution given by 
10 -10 
——(l-¢ ~). 
101 ‘ 


If [= i cos*x dr, compute 


(05) 4) (5-4) 


Verify the error estimate (5.67) for problem 40. 
Use adaptive quadrature to evaluate the integral 
2.0 1 
| sin — a 
VIO 
to within an accuracy «=0.001. 
Evaluate the following double integral 
2 4 
[ [ce -0+ day 
“2 0 
by using Simpson's (1/3)-rule. 
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Matrices and Linear 
Systems of Equations 


6.1 INTRODUCTION 


Matrices occur in a variety of problems of interest; for example, in the 
solution of linear algebraic systems, solution of ordinary and partial differential 
equations, and eigenvalue problems. The matrix notation is convenient and 
powerful in expressing basic relationships in fields like elasticity and electrical 
engineering. In this chapter, we introduce the matrices independently although 
they can be treated, more conveniently, through the theory of linear 
transformations. We assume that the reader is familiar with the concept of 
a determinant and its properties and we describe briefly some simple properties 
of matrices which will be used in the solution of linear algebraic systems 
to which some considerable attention will be given in the later sections. The 
eigenvalue problem will be discussed in Section 6.5, whereas Section 6.6 
will be devoted to a discussion of the singular value decomposition of 
matrices. The theorems will be stated without proof. 


6.2 BASIC DEFINITIONS 


A matrix is an array of mn elements arranged in m rows and nm columns. 
Such a matrix 4 is usually denoted by 


a) a3 Be An 
re i 2 so ie =[a,], (6.1) 
Sami Am? sia ny 
240 
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where a)),)9,... are called its elements and may be either real or complex. 
The matrix 4 is said to be of size (mn). 


lf m=n, the matrix is said to be a square matrix of order n. Thus, 


by db. bs 
B=|b, by by 
by, Bn 


is a square matrix of order 3. We may also have single-row or single-column 
matrices. These are called vectors. Thus, [a;), a2, 4)3,-...d,] is a single- 
row matrix or a row vecfor, and 


ay 
a3) 


a 


is a single-column matrix or a colwmn vector. 

The elements a, in a square matrix form the principal diagonal (or 
main diagonal). Their sum aj; + a) +++++4,,, is called the trace of A. If all 
the elements of a square matrix are zero, then the matrix is said to be a null 
matrix. Thus, if a, =0 for i, j=1,2,...,, then 4 is a null matrix of order n. 
On the other hand, if only the elements on the main diagonal are nonzero, 
then the matrix is said to be a diagonal matrix. For example, 


2 0 0 
C=10 3 0 
0 0 4 


is a diagonal matrix. 
In particular, the diagonal matrix 


0 0 
0 I 0 
0 0 ] 


in which all of the diagonal elements are equal to one, is called a writ matrix 
of order 3. Unit matrices are usually denoted by J. 

A square matrix is said to be an upper-triangular matrix if a, =0 for 
i> j, and a lower-iriangular matrix if a, =0 for i<j. For example, 


2 3 64 
A=|0 5 6 
0 oO 7 
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iS an upper-triangular matrix, and 


z2 0 0 
B=|3 4 0 
5 6 7 


is a lower-triangular matrix. Matrices of the type 
Hy 2 0 0 

ei a 
i) rs 
0 O G43 gy 


are called tridiagonal matrices. A square matrix A in which a, =a, is said 
to be symmetric, if aj;=—a,;, it is said to be skew-symmetric. For example, 


2 5 6 
Ad=|5 8 7 
6 7 4 
is a ‘symmetric matrix,’ and 
0 2 3 
B=|-2 0 4 
3 -4 O 


is a ‘skew-symmetric matrix.’ 
Every square matrix 4 is associated with a number called its determinant, 
which is written as 


| a2 Ay, 

3) ay Fy 
|.A|= 

| n> ace) Onn 


The minor M, of the element aj of |A| is that determinant of order (» -1) 
obtained by deleting the row and column containing aj. The cofactor A, of 
ay is given by 

Ay =(~1)! My. (6.2) 


If |.4|#0, then A is said to be a nonsingular matrix; otherwise, it is said to 
be singular. Thus, 


iI 2 9 
A=|3 4 QO 
5 6 9O 


is singular since |.A|=0. 
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6.2.1 Matrix Operations 


Equality of two matrices Two matrices are said to be equal if they are 
of the same size and if their corresponding elements are equal. 


Addition and subtraction of matrices Two matrices of the same size can 
be added or subtracted by adding or subtracting their corresponding elements. 


Thus, if 
a-(§ i ond a=: | 
5 2 3 #1 


6 4 2 z 
= d -~A= 
A+B | an A-B b i} 


Multiplication of a matrix by a scalar It 


then 


a] a3 ayy 
A=|a, ay ay; 
ay] 433 53 


kay kay kay; 
KA=| Kay, ayy hat 
Ka;, ayy yy 
where & is scalar. 
The following properties of matrices easily follow from the definitions: 
(i) A+(8+C)=(4+ B)+C 
(ii) A+B=B+A 
(iii) A(d+B)=k4+kR, & being a scalar. 
(iv) (kj +k))d4=kAthA, 4 and k, being scalars. 
Multiplication of a matrix by another matrix Two matrices A and B can 
be multiplied only if the number of columns of 4 is equal to the number 


of rows of B. Thus, if 4 and # are of sizes (2*3) and (3x2), respectively, 
then their product C, given by 


C= AB, 


is defined, and will be of size (2x 2). The elements of C are obtained by the 
rule that the element Cy of C is equal to the sum of the products of the 
corresponding elements of the ith row of 4 by those of the jth column of 
B. In general, if A is of size (/xm) and B is of size (mx an), ie. if 
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We find that 
0 -2 = 0 2 3 
A=] 2 0 —§}=-| -2 0 5 | =A. 
c 5 0 =3 =5 0 


It should be noted that this is a ‘skew-symmetric matrix.’ 
Example 6.4 Express the matrix 


I i 8 
A=|6 z 4 
Bs) 4 3 


as the ‘sum of a symmetric’ and a ‘skew-symmetric matrix.’ 


In general, we can write A as 


Now 


which shows that D is skew-symmetric. 
For the example given above, we have 


247 


| 6 5 
A'=|7 2 4 
8 9 3 
Hence 
| 6.5 6.5 
=< =| 6.5 2 6.5), which is a symmetric matrix, 
6.5 6.5 3 
and 
; 0 0.5 15 
a= -0.5 0 2.5|, which is a skew-symmetric matrix. 


-1.5 -2.5 0 
The reader may verify that their sum is equal to A itself. 


Presented By: http://www.ebooksuit.com 


248 CHaPTeR 6: Matrices and Linear Systems of Equations 
6.2.3 The Inverse of a Matrix 


Let 4 be a nonsingular square matrix of order mn. Let B be another square 
matrix of the same order such that 


BA=T, 
where / is the unit matrix of order mn. Then # is said to be the inverse of 
A which is written as 47! so that 
AA ea AKT, (6.4) 


The following fen properties can be shown to hold on the inverse of a 
square matrix: 


(i) A”! exists if and only if |.4|#0. If |.4|=0, A is said to be a singular 
matrix. 


(ii) If A” exists, it is unigue. 

(iii) If A exists, |A7'|=|4'=1// A]. 
(iv) (a 'y's4 

(vy) (a4yt=(a'y, 

(vi) (4By'= Ba 


(vii) If A is a diagonal matrix with diagonal elements a, then 4~' is also 
a diagonal matrix with diagonal elements 1/a,. For example, if 


aj 0 0 
A=|0 a Of, 
0 0 a33 
then 
Vay 0 0 
At=|} 0 Vay 0 
0 0 Vay; 
(viii) Fal, 


(ix) The inverse of an upper-triangular matrix is also an upper-triangular 
matrix. For example, if 


2 3 4 
A=|O 5 Gi, 
0 o 7 
then 
35-21 -2 
Atel 14 = -12 
70 
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Similarly, the inverse of a lower-triangular matrix is also a lower- 
triangular matrix. 


The inverse A~', when it exists, can be computed, as follows 


Ay A, a Any 
ate tl ie (6.5) 
Ain Ay, _ Ary, 


where 4)), 4)2, ... are the cofactors of a), a)3, ... in the determinant 
of the transpose A’ of 4. The matrix on the right side of (6.5) is 
called the adjoint of A. This is not an efficient method for the 
computation of the inverse. A better method is the ‘Gaussian elimination 
method’ (see Section 6.3.2). 


For a nonsquare matrix also, it is possible to define an inverse, called 
the generalized inverse. However, in this book, we shall consider inverses 
for square matrices only (see Section 6.6). 


Example 6.5 Find the inverse of the matrix 


Hence 


5 -2 4 
A=|-2 l l 
4 i 0 
We have |.4|=-—37, and 
5 se a 
A’ =| -2 1 I|=4 
4 1 Oo 
=| 4 -6 
Ate-— -16 -13 
4 -13 ! 


The reader should verify that 447! =/. 


6.2.4 Rank of a Matrix 


Consider a square matrix of order nm. Of the m rows and mn columns, if there 
are at least £ rows and & columns which must be deleted in order to obtain 
a nonvanishing determinant, then the order of the highest ordered nonvanishing 
determinant in A is given by r=n—&, and this number is defined as the 
rank of A and is written r(4). Hence, the rank of a matrix is equal to the 
order of the highest ordered nonvanishing determinant in A. It follows, 
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therefore, that for a nonsingular square matrix of order n, the rank is equal 
to mn. To determine the rank of a matrix, we have to find the order of the 
highest ordered nonvanishing determinant. This method, although general, 
would be tedious when applied to matrices of higher order, for which the 
‘Gaussian elimination method,’ to be described in Section 6.3.2, would be 
particularly suitable. 


Example 6.6 


ol r(.4)=0, since all the elements are zero. 


l 
4 r(A)=2, since | A] #0. 


2 
5 #0; hence r(4)= 2. 


(c) [i | |4|=0; hence r(4)=1. 
: 2 


pe 
a 
an 
ES 
Ml 
= 
= 
— 


(e) d=/4 4 8 |. |Aj#0; hence r(4)=3. 
l 


tad 


6.2.5 Consistency of a Linear System of Equations 
Consider the system of m linear equations in » unknowns: 
Gy pXy + DX, +22 + Ay Xy = d, 


4X + 2z2%q + +++ + yy X, = Dy 


(6.6) 
Any Xy + Ay pho +778 + Ag Xp = Oy. 
The matrix 
ay] ay Ay 
re cy) oo 477, 
| Any? = Onn 


is called the coefficient matrix, and the matrix defined by 
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ay ayo yD 

ic a aoe ee 
(4,b)=) 0 2 oi 

lel ng? ~ Ogun Oy 


is called the augmented matrix. If r(A) is the rank of 4 and r(A, 6) that of 
(4, 5) then the following theorem is proved in books on linear algebra (for 
example, see W.L. Ferrar’s Algebra). 


Theorem 6.3 If r(A)<r(A, 5), then the equations defined by (6.6) are 
inconsistent and there will be no solution; if r(4)=r(A, 6), the equations 
are consistent and there exists at least one solution to the system (6.6). 


Example 6.7 Examine for consistency the equations 


2x-3y4+5z=1 
3xy+y-z=2 
x+4y—-6z=1. 
We have 
2 3 > 
A=|3 | =!). 
l 4 -§ 
Then 
|4|=0, but 5 #0; hence r(A4)=2. 
Further 


2 -3 5 | 
(A, b) =} 3 l | 2), 
| 4 —-6 | 
and it can be seen that all determinants of the third order formed from 


(A, b) are zero and that r(A,b)=2. It follows, therefore, that the equations 
are consistent. 


Example 6.8 Find whether the following system is consistent 
x—-4y+52=8 
3x+7y-2=3 
x+l5y—Ilz =—-]4., 
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12 
xtle=Viml? +1 PF +-+1x, 2 = Sia =|jxij, 6-12) 


i=] 


| Xl = max |x)| (6.13) 


The norm ||+||, is called the Euclidean norm since it is just the formula for 
distance in the three-dimensional Euclidean space. The norm ||+||,. is called 
the maximum norm or the uniform norm. 

It is easy to show that the three norms ||x|l,,|/x||, and ||X||.. satisfy 
the conditions (6.7) to (6.9), given above. Conditions (6.7) and (6.8) are 
trivially satisfied. Only condition (6.9), the triangle inequality, needs to be 
shown to be frwe. For the norm ||x||, we observe that 


A 
Ixt+yll= > Ix +! 


Fol 


<) ele D 


= 
- ft 

=> ixl+ >} lil 
i=l i=] 


=[nlh +iy ll (6.14) 
Similarly, for ||X||... we have 


NZ +y¥ loo = max | x; +»! 
smax (| x;|+]¥;|) 
ie ess (6.15) 
= (lla + 11Y Ilee « 
The proof for the Euclidean norm is left as an exercise to the reader. 


To define matrix norms, we consider two matrices A and & for which 
the operations A+ and AB are defined. Then, 


|A+ B|s| A|+| Bl (6.16) 
| AB) sj || B| (6.17) 
la@A|=|a@||A| (@ a scalar). (6,18) 


From (6.17) it follows that 
|AP|s|Al?, (6.19) 
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where p is a natural number. In the above equations, |.4| denotes the matrix 
A with absolute values of the elements. 


By the norm of a matrix A=| a, |, we mean a nonnegative number, denoted 
by || Aj], which satisfies the following conditions 


|| Aljz0 and |[Al|=0 ifandonly if 4=0 (6,20) 


||aA||=|a|||4||  (@ a scalar) (6.21) 
| A+ Bils|] A+ i Bl (6.22) 
|| AB || s|| Alli Bil. (6.23) 


From (6.23), it easily follows that 
A? SH All’, (6.24) 


where p is a natural number. 


Corresponding to the vector norms given in (6.11)}46.13), we have the 
three matrix norms 


All = | the col 6.25 
ll lh max > Lay! (the column norm) (6.25) 
#2 
Il All. 5 | ay | (the Euclidean norm) (6.26) 
i,j 
| Allo = max >) lay| (the row norm). (6.27) 
j 


In addition to the above, we have ||4||, defined by 


\| Alb = (Maximum eigenvalue of A'4)'?. (6.28) 


The eigenvalues of a matrix will be discussed in Section 6.5. 

The choice of a particular norm is dependent mostly on practical 
considerations. The row-norm is, however, most widely used because it is 
easy to compute and, at the same time, provides a fairly adequate measure 
of the size of the matrix. 

The following example demonstrates the computation of some of these 
norms. 


Example 6.9 Given the matrix 


l 2 3 
A=|4 5 6 
7 8 9 


find || A|l.|] All. and ||A [lao 
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This method is obviously unsuitable for solving large systems, since the 
computation of 4~' by cofactors will then become exceedingly difficult, and 
one should therefore adopt methods which do not require the computation 
of the cofactors. We will describe such methods in the subsections below 
and these can be applied to any number of equations. 


6.3.2 Gauss Elimination 


This is the elementary elimination method and it reduces the system of 
equations to an equivalent upper-triangular system, which can be solved by 
back substitution. 

We consider the system given in (6.29), viz., the system of 7» linear 
equations in » unknowns 


yj Xy + AX + yg Xy Ho + Ay Xy =H, 


yj Xj + Ay7Xq + Ay3X3 + +++ + Og_Xq = By (6.29) 
gp Xy + yg Xo + .y3Xy t+ Any Xp, = by. 
There are two steps in the solution of the system (6.29), viz., the elimination 
of unknowns and back substitution. 
Step J: The unknowns are eliminated to obtain an upper-triangular system. 
To eliminate x, from the second equation, we multiply the first equation 
by (-a3,/a;;) and obtain 


a ba | ~ ay, AL x, ayy 
qj qj 


Adding the above equation to the second equation of (6.29), we obtain 


pees | ee +h he pea | Ie mer oe 21 
c a2 me |r +{ ox a3 |, + ac in) ay Jr b, -—b, a 
(6.32) 


which can be written as 

A'ynk2 + A44%y +++ +:27_X_ = 2, 
where a). = a3 — 4) (4),/a,,), etc. Thus the primes indicate that the original 
element has changed its value. Similarly, we can multiply the first equation 


by —a3,/a,, and add it to the third equation of the system (6.29). This 
eliminates the unknown x, from the third equation of (6.29) and we obtain 


Gyp Xp + yay +++ + By Xy_ = Oh. (6.33) 
In a similar fashion, we can eliminate x, from the remaining equations and 
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after eliminating x, from the last equation of (6.29), we obtain the system 
HX + A 2%y + 2y3X%y +--+ yy Xy = By 
Gg Xq + Ay3X3 +--+ Dy_Xy = Dy 


G32 Xq +.53%3 407+ + Bay Xy = 3 an) 


dX + yyX3 t+ x, = B. 


We next eliminate x, from the last (nm —2) equations of (6.34). Before this, 
it is important to notice that in the process of obtaining the above system, 
we have multiplied the first row by (—a2;/a,)), ie. we have divided it by a), 
which is therefore assumed to be nonzero. For this reason, the first equation 
in the system (6.34) is called the pivot equation, and a, is called the pivor 
or pivotal element. The method obviously fails if a,,=0. We shall discuss 
this important point after completing the description of the elimination method. 
Now, to eliminate x, from the third equation of (6.34), we multiply the 
second equation by (—a;,/a;,) and add it to the third equation. Repeating 
this process with the remaining equations, we obtain the system 


ch Xy + Aya Xy + Gy5Xy ++ a,x, =D, 
Gy7X + 43X3 +--+ + 0),,x,, = By 
44% +--+ ay, = by 


+ 
1 


(6.35) 


On3%q te +X, = By. 


In (6.35), the ‘double points’ indicate that the elements have changed twice. It 
is easily seen that this procedure can be continued to eliminate x; from the 
fourth equation onwards, x, from the fifth equation onwards, etc., till we 
finally obtain the upper-triangular form: 


yy Xy + 2X7 + 4 Xq +0 +, X, = 5 
Gg Xz + GygX4 +++-+4),%, = by 
G33%3 +++ G4, ¥, = BY (6.36) 


atx, 25, 


where ai!) indicates that the element a,,, has changed (mn —- 1) times. We 
thus have completed the first step of elimination of unknowns and reduction 
to the upper-triangular form. 


Step 2: We now have to obtain the required solution from the system 
(6.36). From the last equation of this system, we obtain 
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Bor) 


xy ae 

This is then substituted in the (n—1)th equation to obtain x,_, and the 
process is repeated to compute the other unknowns. We have therefore first 
computed x,, then X,_),%,-25-+-.¥2, 2), in that order. Due to this reason, the 
process is called back substitution. 

We now come to the important case of the pivot being zero or very 
close to zero. If the pivor is zero, the entire process fails and if it is close 
to zero, round-off errors may occur. These problems can be avoided by 
adopting a procedure called pivoting. If a,, is either zero or very small 
compared to the other coefficients of the equation, then we find the largest 
available coefficient in the columns below the pivot equation and then interchange 
the two rows. In this way, we obtain a new pivot equation with a nonzero 
pivot. Such a process is called partial pivoting, since in this case we search 
only the columns below for the largest element. If, on the other hand, we 
search both columns and rows for the largest element, the procedure is 
called complete pivoting. It is obvious that complete pivoting involves more 
complexity in computations since interchange of columns means change of 
‘order’ of unknowns which invariably requires more programming effort. 
In comparison, partial pivoting, i.e. row interchanges, is easily adopted in 
programming. Due to this reason, complete pivoting is rarely used. 


Example 6.17 Use Gauss elimination to solve 
2x+y+z=10 


3x+2y+3z=18 
x+4y4+9z7=16. 


We first eliminate x from the second and third equations. For this we 
multiply the first equation by (—3/2) and add to the second to get 


y+3z=6. (i) 
Similarly, we multiply the first equation by (—1/2) and add it to the third to get 
dy +17z = 22. (it) 


We thus have eliminated x from the second and third equations. Next, we 
have to eliminate y from (i) and (ii). For this we multiply (i) by —7 and add 
to (ii). This gives 


—4z=-20 or z=5. 
The upper-triangular form is therefore given by 


2x+y+z=10 
y+3z=6 


ze 5. 
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It follows that the required solution is x=7, y=-9 and z=5. 
The next example demonstrates the necessity of pivoting in the elimination 
method. 


Example 6.12 Solve the system 
0.0002x + 0.3003 » = 0.1002 
2.0000x + 3.0000» = 2.0000. 


The exact solution of the system is easily seen to be x =1/2 and y=1/3, 
We first solve the system without pivoting. Multiplying the first equation 
by (~2/0.0002) and adding it to the second, we obtain 


0.3003 x 2 0.1002 «2 
3.0000 — ——_——— | » = 2.0000 - ——___—_,, 
( 0.0002 ) 0.0002 
which simplifies to 
1498.5y = 499, 


Hence the triangular system is 
0.0002x + 0.3003 y = 0.1002 
1498.5y = 499. 
The solution to the system is given by y=0.3330 and x =0.5005; the errors 
in the solution being due to the large multiplier. 
We next interchange the two rows so that the system is written as 
2.0000x + 3.0000» = 2.0000 
0.0002x + 0.3003 y = 0.1002 


Multiplying the first equation by (—-0.0002/2) and adding it to the second, 
we obtain 


(0.3003 > 3.0000 x a = 0.1002 - 2™ 0.0002 
2 2 
which simplifies to 
0.3000» = 0.1000. 
Hence the solution is 
y “3 and x=— 


6.3.3 Gauss—Jordan Method 


This is a modification of the Gauss elimination method; the essential difference 
being that when an unknown is eliminated, it is eliminated from all equations. 
The method does not require back substitution to obtain the solution and is 
best illustrated by the following example: 


Presented By: http://www.ebooksuit.com 


Hidden page 


Presented By: http://www.ebooksuit.com 


Hidden page 


Presented By: http://www.ebooksuit.com 


Section 6.3: Solution of Linear Systems—Direct Methods 263 


Example 6.14 We shall consider again the system given in Example 6.13. 
We have here 


2 I I 
Aewl3 = 62 
l 4 9 
The augmented system is 
2 1 lo: oJ o O 
3 2 3 0 l 0 
1 4 9 0 0 I 


2 l 1 l 0 0 
0 1/2 3/2 —3/2 l 0 
0 7/2 17/2 =/2 0 l 


Finally, at the end of the second stage, the system becomes: 


2 } l : l 0 oO 
0 V/2 3/2 = 3/2 l 0}. 
0 0 -2 : 10) -7 ] 


0 1/2 3/2 <2 |, 
0 0 -2 10 
2 l ] 0) 
0 1/2 3/2 I 
0 0 -2 -/] 
and 
2 l 1 : 0 
0 L/2 3/2 : 0}, 


0 0 2: l 


whose solution by back substitution yields the three columns of the matrix: 
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-3 W2 =I/2 
2 = =-I7/2 3/2), 
=§ 2 1/2 


which is the required inverse 47!. 
We can also find 


|A|= a3 Ja =-2 


by looking at the triangulated coefficient matrix. If this value is zero, then we 
cannot back substitute and the matrix has no inverse. 


6.3.5 Number of Arithmetic Operations 


Since the total execution time depends on the number of multiplications and 
divisions in Gaussian elimination, we give below a count of the total number 
of floating-point multiplications or divisions in this method. 

For eliminating x), i.e. in Eq. (6.32), the factor a,,/a,, is computed 
once. There are (—1) multiplications in the (n—1) terms on the left side and 
1 multiplication on the right side. Hence the number of ‘floating-point’ 
multiplications/divisions required for eliminating x, is 1+n-—1+l=n+l1. 
But x, 1s eliminated from (1-1) equations. Therefore, the total number of 
multiplications/divisions required to eliminate x, from (n-1) equations is 

(n-1)(n+1) =(n—-1) (n+2-1). 


Similarly, the total number of multiplications/divisions required to eliminate 
x, from (n~2) equations is 
(n—2)n=(n—-2)(n+2—-2). 


The total number of multiplications/divisions required to eliminate x, from 
(n-3) equations is 


(n-—3) (mn —1) =(n-3) (n+ 2-3). 


Similarly, the total number of multiplications/divisions required to eliminate 
x, from (— p) equations is 


(n— p)("+2— p), 
and finally, x,_, is eliminated in 
[ne -(n —1)} [n+ 2 -(n —1))=1.3. 


Summing up all the above, we can write the total number of arithmetic 
operations (i.e. multiplications/divisions) as 
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rs ed | 


>, (n= p)(n+2-p)= >" ((n- py +2(n- p)] 


p=l 


a-| 
= > (ni? +p ~2np+2n—-2p) 
p=l 


aye fee s-261 ie m=Dn 


+ In(n -1)-22—™" 


Pe 
t=, 
3 
where we have used the formulae: 
ee ee ae, and a _ n(ntl) (ntl) | 
2 6 


It follows that the total number of ‘floating-point’ multiplications or divisions 
in Gaussian elimination is m°/3. In a similar way, it can be shown that the 
Gauss—Jordan method requires °/2 arithmetic operations. Hence, Gauss 
elimination is preferred to Gauss—Jordan method while solving large systems 
of equations. 


6.3.6 LU Decomposition 


This method is based on the fact that a square matrix A can be factorized 
into the form LU, where £ is unit lower triangular and C/ is upper triangular, 
if all the principal minors of A are nonsingular, i.e. if 


ay) a2 a3 
#0, |G, dp ayy / #0, etc. 


ay) 447 43 
It is a standard result of linear algebra that such a factorization, when it 
exists, 15 unique. 
We consider, for definiteness, the linear system 

Gy xy + Gy2Xz + QyyX3 = 4 

Gy 1X, + Gy2%2 + Gy4%y = Dy 

Gy X + 43% + Gy3X3 = dy, 
which can be written in the form 

AY =B. (6.38) 


a] 2 


a,, #0, | 
a) a9 
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Let 


A=LU, (6.39) 
where , 
| 0 0 
L = by l 0 (6.40) 
and 


U=|0 ps (6.41) 


Hence (6.38) becomes 


LUX = B. (6.42) 
If we set 

UX =Y, (6.43) 
then (6.42) may be written as 

LY=B, (6.44) 
which is equivalent to the system 

yn =5 
lV + ¥2 = 52 


Bi + hay2 + ¥3 = 8 
and can therefore be solved for y), 2, 3 by the forward substitution. When 
Y is known, the system (6.43) becomes 
By hy XT 9X = Hh 
M99 %2 + Uy3%3 = ¥25 
4y3%3 = Y3s 


which can be solved for x,, x2, x; by the backward substitution. 

We shall now describe a scheme for computing the matrices Z and U, 
and illustrate the procedure with a matrix of order 3. From the relation 
(6.39), we obtain 


PE QO 0 fim, mp. qj, 42 a3 
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Multiplying the matrices on the left and equating the corresponding elements 
of both sides, we get 


My =, = HD =A, My = AQ 
boyy) = Gay, Agyyg + QQ =A, yg + HQ = Ay (6.45a) 


fyyMy) = 451; Iyytyq + h2ug2 = 453, By yuy3 + balay + 53 = 453 
Solving them, we get 


i = “21. E =—il; 
a aj . ay 
_ 932 — (43 )/a 1 42 


from which uj; can be computed. 
We thus have a systematic procedure to evaluate the elements of £ and 
LU. First, we determine the first row of U and the first column of 2; then 
we determine the second row of U and the second column of L, and finally, 
we compute the third row of UL. The procedure can be obviously generalized. 
When the factorization is effected, the inverse of A can be computed 
from the formula 


Ate(tuy' sur, (6.46) 
Example 6.75 Solve the equations 
2x+3y+z2=9 
x+2y+3z=6 
3x+yp+2z7=8 
by LU decomposition. 
We have 
Z 3 l 
A=|1 2 3}. (i) 
3 l 2 
Let 
l 0 O}) m4, uy My 3 3 l 
by 1 OT] 0 pp gg =f 1028]. (ii) 
by by 14, 0 0 wy} [3 2 
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my =2, M2 =3, 3 =]; 


fy yy} =| so that la) =; 


Iy)m,=3 so that J, =>. 
For #27 and w2;, we have the equations 
Iyymyz +g = 2 and 1y)m3 +423 = 3, 
from which we obtain 
U3 => and w= >. 
Finally, /,) and wu; are obtained from 
IyjMjq +4quaq =] and = yyy +4903 +33 = 2, 


and hence 
jg =-7 and wy; =18. 
It follows that 


1 0 0} 2 3 ] 
A= 1/2 l OVO W2 5/2 (iti) 
3/2 —i 1] 07 0 18 
and hence the given system of equations can be written as 
| 0 Oj} 2 3 1 || x 9 
1/2 ] 0} 0 fz S/2)) yp |=) 6 (iv) 
3/2 -7 1) 0 0 18 || z 8 
OF, as 
1 0 Oy) [9 
1/2 1 O}] 2 |=! 6], (v) 
3/20 O-7 Lijiys] [8 
where 
2 3 l x yy 
0 W2 5/2 Wy |=] y5 |. (vi) 
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Solving the system in (v) by forward substitution, we get 


y =o Fe => yy = 5. 


With these values of y), y2, ¥a, eq. (vi) can now be solved by the back 
substitution process and we obtain 


35 29 5 
=e = 


“18° 1g ~ 18 


x 


6.3.7 LU Decomposition from Gauss Elimination 


We have seen that Gaussian elimination consists in reducing the coefficient 
matrix to an upper-triangular form. We show that the LU’ decomposition of 
the coefficient matrix can also be obtained from Gauss elimination. The 
upper-triangular form to which the coefficient matrix is reduced is actually 
the upper-triangular matrix C! of the decomposition ZU’. Then, what is the 
lower-triangular matrix L? For this, we consider the system defined by 


AX =b, (6.47) 
where 
ah 2 hy x by 
A=!) 42 3 |) XM=)x |, b=| > 
@, 943933 X3 by 


To eliminate x, from the second equation, we multiply the first equation by 
a3,/a,, and subtract it from the second equation. We then obtain 


oe oa MED I oe he 
c ay ts, {a ayy at C 5 2a] 
or 

@5).X7 + a43x3 = 55. (6.48) 


The factor /,;=«,/a,, is called the mu/tiplier for eliminating x, from the second 
equation. Similarly, the multiplier for eliminating x, from the third equation 
is given by (5; =4;,/a,,. After this elimination, the system is of the form 


Ay X, + ayy Xy + 33 = Oy 
4X4 + G44Xy =b, (6.49) 
G37 + 33%3 = by. 


In the final step, we have to eliminate x, from the third equation, For 
this we multiply the second equation by aj,/a;, and subtract it from the 
third equation. We then obtain 
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and 
-73 78 24 
92 66 25 | =-118.94. 
80 37 10.01 
Example 6.18 The system 
10x, + 7x +8x, + 7xy =32 
7X, +5x2 + 6x3 + 5x, = 23 
Sx, + 6x, +10x, + 9x4 =33 


(i) 
Tx, + 5x, +9x, +10x5 =31 
is ill-conditioned since the system 
10x, + 7x5 + 8x, + Txy = 32.1 
7X; + 5x, + 6x, + 5x4 = 22.9 
8x, + 6x, + 10x, + 9x4 =32.9 
7x, + 5x + 9x, +10x4 =31.1 


has the solution [6, -7.2, 2.9, -0.1] whereas the system (i) has the solution 
(1, 1. 1, 1). 


Ill-conditioning can usually be expected when |.4|, in the system Ax =b, is 
small. The quantity v(.4) defined by 


v(A)=|| Al] A |], (6.65) 


where || 4]! is any matrix norm, gives a measure of the condition of the 
matrix, It is called the condition number of the matrix. Large condition 
numbers, as a rule, indicate ill-conditioning of a matrix. We give below 
examples of ill-conditioned and well-conditioned matrices. 


Example 6.29 Let 
z tf 
A= 
; al 


Taking the Euclidean norms, we obtain 
jj 4\|, =3.165 and Ce le = 158.273. 
Hence v(.4)= 500.974. It follows that 4 ts ‘1ll-conditioned.’ 


Example 6.20) Let 
-0.6 0.6 
B= ; 
0.4 0.2 
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We have 


Bll. =0.959 and || ' ||, =2.664, 
Hence 
v(B)=|| Bl]. Bo |. = 2.555. 


It follows that B is a well-conditioned matrix. 
Another indicator of ill-conditioning is the following. If A=[a,] and 


§; =(a" +a Silat ay? (6.66) 
then the quantity 
p-—l4l (6.67) 
S$) Sp -..5, 


indicates, in some sense, the smallness of the determinant of A. If & is very 
small compared to 1, then the matrix 4 is ‘ill-conditioned.’ Otherwise, It is 
well-conditioned. For the matrix A in Example 6.19 above, we obtain | 4 | = 0.02, 
5, = [5 = 2.2360679 5, = 2.240 and & =3.993x10~. Similarly, for the matrix 
B in Example 6.20, we obtain | B|=-0.36, s, = ,/0.72 = 0.848, s) = 0.447 and 
k = 0.950. 


6.3.11 Method for Ill-conditioned Matrices 


One method of improving the accuracy for an ill-conditioned system is by 
means of working all the calculations to more number of significant digits. 
But multilength arithmetic is time-consuming and therefore uneconomical. 
One possible alternative is to improve upon the accuracy of the approximate 
solution by an iterative procedure. This is described below. 
Let the system be 
Gy 1X + 2 Xy + GygX3 = by 
71%) + Ay7Xy + 4y4%3 = by (6.68) 
31%, + 2y2.%2 + 444Xy = dy. 


Let x, %),%3 be an approximate solution. Substituting these values on the 
left-hand side, we get new values of 5, 5),;, say, b,,5),5,. Thus the new 
system is 


Gj X + Ay2X_ + ayyXy = 
Gy) X) + yy Xy + G4 Ky = 


4) X, + 247.0 + 43%, = 


(6.69) 


a 


= 
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Subtracting each equation in (6.69) from the corresponding equation in (6.68), 
we get 


A pe, + ay 2€y + a) 3e3 =a 
Gz )€ + Ay,€3 +4z3e; =a, (6.70) 
4 )¢| + ay7€7 + f43¢, = dy, 


where ¢, =x,-%, and qd, =b, —&. We now solve the system (6.70) for ¢,,e, 
and €;. Since ¢; =x; —X,, we obtain 

x =e,+ 3%, (6.71) 
which is a better approximation for x;.The procedure can be repeated to 
improve upon the accuracy. 


6.4 SOLUTION OF LINEAR SYSTEMS—ITERATIVE METHODS 


We have so far discussed some direct methods for the solution of simultaneous 
linear equations and we have seen that these methods yield the solution after 
an amount of computation that is known in advance. We shall now describe 
the iterative or indirect methods, which start from an approximation to the 
true solution and, if convergent, derive a sequence of closer approximations— 
the cycle of computations being repeated till the required accuracy is obtained. 
This means that in a direct method the amount of computation is fixed, 
while in an iterative method the amount of computation depends on the 
accuracy required. 

In general, one should prefer a direct method for the solution of a linear 
system, but in the case of matrices with a large number of zero elements, 
it will be advantageous to use iterative methods which preserve these elements. 

Let the system be given by 


yp + yyy + yyy to +X, =D 
hy Xy + Ag Xy F343 to + Ay_Q Ky, = by 


(6.72) 
31% + Gy9Xy +.043X3 +++ + Oy,X, = by 


Byj%y Fy 7%3 + I | tet yy Xt, = b, 


in which the diagonal elements a, do not vanish. If this is not the case, 
then the equations should be rearranged so that this condition is satisfied. 
Now, we rewrite the system (6.72) as 
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pee sae ee 
Hy Fy ay 
ay, ay 479 

y, a. Sly Sty -. 
G33 (943 33 
one: eae ee 
Onn Fn Gan 

Suppose x, xy", = 


X> Xz, ...) X,+ Substituting in the right side 
second approximations 


2&9 
Hy ay 

x) 22 _ 1 _., 
ay ay 

2 a3) fl 

7) = 221M) 
433 933 


x2) = Pn _ Sal A 
rary Onn 
Similarly, if x(”, xf”... 


next approximation is given by the formula 
xi) DL _ 2 fm _.,, 
hy Fy 
hm) On _ 21 sn) _ 
42 ay 


xe) Bn St 
i Onn Onn, 


If we write (6.73) in the matrix form 
A=BXY+C 
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eos om 


bes oe 


(6.73) 


AA 


, x) are any first approximations to the unknowns 


of (6.73), we find a system of 


Ain A), 
i I 


92m (l) 

a4 n 

6.74 
Bin ee 
133 


2, 


ad 
AW] (1) 
ee xn-|* 
WuFT 


x” are a system of nth approximations, then the 


Sin in) 

foi a 
cy | 
S20 fn) 
a2 (6.75) 
ay n=l (nj) 
a) yim) | 

rT 
(6.76) 
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then the iteration formula (6.75) may be written as 


XO) = ax) +. (6.77) 


This method is due to Jacobi and is called the method of simultaneous 
displacemenis. It can be shown that a sufficient condition for the convergence 
of this method is that 


| Bil<1. (6.78) 


A simple modification in this method sometimes yields faster convergence 
and is described below: 

In the first equation of (6.73), we substitute the first approximation 
(el? x a xf). xs ‘') into the right-hand side and denote the result as xt 
In the ceabial ‘eaentlon: we substitute (xi, ae Be vei x") and denote 
the result, as xe In the third, we substitute (x2, x? : xo i xf) and 
call the result as x!), In this manner, we complete the first stage of iteration 
and the entire process is repeated till the values of x,, x;,...,x, are obtained 
to the accuracy required. It is clear, therefore, that this method uses an 
improved component as soon as it is available and it is called the method 
of successive displacements, or the Gauss—Seidel method. 

The Jacobi and ee, methods converge, for any choice of the 
first approximation x! j Dey j=1,2,...,2), if every equation of the system (6.73) 
satisfies the condition that the sum of the absolute values of the coefficients 
a,/a, i8 almost equal to, or in at least one equation less than unity, Le. 
provided that 


it 


2 


jul, pei 


= (f=1, 2..." (6.79) 


where the ‘<" sign should be valid in the case of ‘at least’ one equation. It 
can be shown that the Gauss-Seidel method converges twice as fast as the 
Jacobi method. The working of the methods is illustrated in the following 
example: 


Example 6.21 We consider the equations: 
10x) — 2x2 — 3 —-x4 =3 
—2x, + 10x, —x,—x4=15 
—x, — xX, +10x, -— 2x, = 27 
—X — xX) — 2x, + 10xy = 9. 


To solve these equations by the iterative methods, we re-write them as 
follows: 


=0.3+0.2x, + 0.1x; + 0.1x, 


Xx, =1.5+0.24, +0.1x, + 0.1%, 
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x; = 2.74 0.1x, + 0.Lx, +0.2x, 

x4 = 0.94 0.1x, + 0. 1x5 + 0.224. 
It can be verified that these equations satisfy the condition given in (6.79). The 
results are given in Tables 6.1 and 6.2: 


Table 6.1 Gauss—Seidel Method 


n Mi Ky Xy Xy 

1 0.3 1.56 2886 -0.1368 
2 0.8869 1.9523 2.9566 -0.0248 
3 0.9836 1.9899 2.9924 -0,0042 
4 0.9968 1.9982 2.9987 -0.0008 
5 0.9994 19997 2.9998  -0.0001 
6 0.9999 1.9999 30 0.0 

7 1.0 2.0 3.0 0.0 

Table 6.2 Jacobi's Method 

n x My Xs Ka 

1 0.3 1.5 27 -0.9 

2 0.78 1.74 27 -0.18 
3 09 1.908 2916  -0.106 
4 0.9624 19608 2.9592 -0.036 
5 0.9845 1.9848 2.9851 -0,0158 
6 0.9939 1.9938 2.9938 0.1006 
7 0.9975 19975 2.9976 0.0025 
8 0.9990 1.9990 2.9990 -0.0010 
9 0.9996 19995 2.9996 -0,0004 
10 0.9998 19998 29998 -0.0002 
11 0.9999 1.9999 2.9999 =—0,0001 
12 1.0 20 3.0 0.0 


From Tables 6.1] and 6.2, it is clear that twelve iterations are required by 
Jacobi's method to achieve the same accuracy as seven Gauss-Seidel iterations. 


6.5 -THE EIGENVALUE PROBLEM 


Let A be a square matrix of order n with elements aj. We wish to find a 
column vector XY and a constant 4 such that 


AX = AX. (6.80) 
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The characteristic equation of this matrix is given by 
S=A 0 1 
0 ~2-A 0 |=0, 
] 0 5=<A 


which gives 4, =-2, A, =4 and A, =6. The corresponding eigenvectors are 
obtained thus 


(i) A,=-2. Let the eigenvector be 


xy 


X, =| X2 |. 
xy 
Then we have: 
x S| 
Al xq |= 2] x, |, 
xy *3 


which gives the equations 
7x, +x3=0 and x,+7x, =0 


The solution is x, =x, =0 with x, arbitrary. In particular, we take x, =1 and 
the eigenvector is 


0 
X,=| 11. 
0 
(ii) a, =4. With 
| 
Xy =| x 
*3 


as the eigenvector, the equations are 
%+%,=0 and -§x, =0, 
from which we obtain 
M=-x, and x, =0, 
We choose, in particular, x, =1/,/2 and x; =—1/,/2 so that . +x? +x} =]. 
The eigenvector chosen in this way is said be normalized. We therefore have 


Vf2 
Oo |. 
Wf2 


X= 
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from which we see that 
2.3 
x) =| 1 
0 


and that an approximate eigenvalue is 3. 
Repeating the above procedure, we successively obtain 


2.1 22 2 2 2 
4/11); 4¢11); 441); 421); 410i. 
0 0 0 


0 0 
It follows that the largest eigenvalue is 4 and the corresponding eigenvector is 
2 
Lh 
0 


6.5.1 Eigenvalues of a Symmetric Tridiagonal Matrix 


Since symmetric matrices can be reduced to symmetric tridiagonal matrices, 
the determination of eigenvalues of a symmetric tridiagonal matrix is of 
particular interest. Let 


a 42 0 
Ay=| 42 Gyn 43, |. (6.84) 
0 a3 ay 
To obtain the eigenvalues of 4,, we form the determinant equation 
a,-A a2 0 
IAI) a a2 —A ay, |=0. 
0 ayy ay,—A 
Suppose that the above equation is written in the form 
¢,(A) =0. (6.85) 
Expanding the determinant in terms of the third row, we obtain 
KnstGa=al ? a 
12 ay, —A a2 a73 
= (053 — A) (4) — 93 (a); — 4) ag; 
= (433 - 2) (A) -23,4 (A) 
= 0. (6.86) 
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qy Ay ayy 
A= > Gy Gy, |- (6.93) 
ay 433 a3 


Householder’s method consists in finding a real symmetric orthogonal matrix 
P, such that 


aq; a2 O 
PAP|=| a2 a3) 4; |, (6.94) 
0 a 4 


where the primes denote that the elements have changed. Householder suggested 
that P, should be of the form 


P,=1-27V', (6.95) 
where 
V=[0 vy vj’ and pty oy, (6.96) 
It is easily verified that 
1 0 0 
P,=|0 1-2vh -2vg¥, |, (6.97) 
0 -2r7; 1-2¥ 
where 
ve tue =1. (6.98) 
Further, 


RA =(-2VV" jy -2v'y! 
=(1-2¥V")(/-2¥¥') 
=]-4VV' +4py' py! 
= J. (6.99) 


We thus see that P, is both symmetric and orthogonal. By direct multiplication, 
we find that 


ay a (1 -2¥3)— 243%); —2a,2¥¥3 + a3(1 - 2v5) 
AP, =| ayy ay) (1— 29) - 2ay3vqv3_ —2aq9¥2¥3 + a3 (1 - 209) 
@3 yz (1— 2v9)— 2a43Vyv3_ —2ay3¥2¥4 + 453(1— 2v5) 
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and therefore 


1 0 0 
PAR =|0 1-23 -2yyv, 
QO =2y», 1-24 


ay, yy (1—2v5)—2ay3vyvy  ~2a,2vQV5 + a43(1 - 2¥5) 
x] yy Gy (1— 2V9)— 2agg¥y¥3 -2adyVQV3 + 4y3(1-2¥$)|. (6.100) 
Gy3 ayy (1-29) - 2ay3Vyv3  -2a93Vy¥5 + 453(1— 245) 
Comparing (6.100) with (6.94), we find that 
0 = -2ay2v9V3 + a3 (1- 2v§) 
= a3 — 2V3(a)2¥2 + 2,343) 
= a, -274, (6.101) 
where 
F = G22 + )3V3. (6.102) 
Also, 


at, = a2 (1- 2v3) —2a,3r)¥4 


= Gy — 22 (a 2V2 + 4345) 
oe (6.103) 


using (6.102). From (6.101) and (6.103), it follows that 
(a}p)* = (a2 - 27)? + (qy3 - 25)? 


= ah, + apy +40? (vd +04) - 4r(ajqvy + 44345) 


= ay +444, (6.104) 
using (6.98) and (6.102). Hence, 
aly = ty af, + af; =ay) -2rv, =45S, say, (6.105) 
We therefore have two equations, viz., 
a3 — ary, = 0 (6.106) 
and 
a —2rv, = +5, (6.107) 
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Re Fi Faas PAA (6.114) 
To complete the construction, we define the orthogonal matrix 
g'=P_P_... BP (6.115) 


so that 4, = QR as required. The sequence {A,} converges either to a triangular 
matrix with the eigenvalues of A on its diagonal, or to a near-triangular matrix 
from which also the eigenvalues can be easily calculated. Since the sequence 
converges slowly, a technique, known as shifting, is used to accelerate the 
convergence. This technique will not be discussed here, and the interested 
reader may refer to advanced texts for this. 


6.6 SINGULAR VALUE DECOMPOSITION 


We have so far considered square matrices only and in Section 6.3.6 we obtained 
the LU decomposition of a square matrix. A somewhat similar decomposition 
is also possible of a rectangular matrix and this is called the singular value 
decomposition (SVD). The SVD is of great importance in matrix theory 
since it is useful in finding the generalized inverse of a singular matrix and 
has several image processing applications. 

Let 4 be an (mxn) real matrix with mn. Then the matrices 4'4 and 
AA' are non-negative, symmetric and have identical eigenvalues, say A,. We 
ean then obtain the n orthonormalized eigenvectors, say x,, of 4'4 such that 


A! Ax, =A,%,. (6.116) 


If we assume y,, to be the n orthonormalized eigenvectors of 44", then we 
have 


AA y, =A,Yp- (6.117) 
Then A can be decomposed into the form 
A=UDF', (6.118) 
where 
u'u=V'y=pv' =], (6.119) 
and 


D=diag (,[4,,.f Ap, --sf An). (6.120) 


The decomposition defined by (6.118) is called the singular value decomposition ~ 
of A. The matrix (nxn) consists of x, which are the » orthonormalized 
eigenvectors of 4'4. It follows then that the eigenvectors of 4d", i.e. y, are 
given by 


¥, =x, (6.121) 


Ta, 
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The matrix D is a diagonal matrix given by 


j4 0 0 
pe) ° VRo~ 129 


e OO fa ge 
where VA. ; Jay aes af hy are called the singular values of A and are 
such that 


«fay 2 fag 2-2 fa, 20. (6.123) 
If the rank of A is r<a”, then 


al Boag af Apgg fon yf, = 0. (6.124) 


It can be shown that the singular value decomposition of A is unique if the 
A, are distinct and (6.123) is satisfied. In case, 4 is a square matrix of 
order n, then the matrices U, D and F are also square matrices of the same 
size and the inverse of A can be trivially computed, since 

At =yp"yt (6.125) 
and 


= : l l l 
D = ie Ses zr} (6.126) 
Yar va Vay 
If any of the A,’s are zero, then the matrix A is singular. Similarly, if the A, 
are very small, then the matrix 4 is very nearly singular. Thus the singular- 
value decomposition of a matrix gives a clear indication whether the matrix 


is singular or very nearly singular. The following example demonstrates the 
method. 


Example 6.25 Obtain the singular-value decomposition of 


ies? 
A=|1 1}. 
| 3 


We have 


111 3 6 
T T 
A = and A A= 
b i b | 
The eigenvalues of A'A are given by 4, =16,64 and A, = 0.36. For the corres- 
ponding eigenvectors, we have 
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fs wbalselah 


which gives the system: 
3x, + 6x = 16.64x, 
6x, + 14x, =16.64x,. 
The solution is given by 


Again, we have 


vale} 


which gives the system 
3x, +6x, = 0.36x, 


6x, + 14x, =0.36x. 
The solution is 


We also have ,/4, = 4.080 and ,/4, = 0.60. 
The eigenvectors of AA‘ can then be obtained from (6.121): These are 
given by 


0.5480 0.1833 
0.3235} and | 0.8555 |. 
0.7727 —0.4889 


The singular-value decomposition of 4 is then given by 
1 2} | 0.5480 0.1833 


Az=|1 1)=/0.3235 0.8555 ka . bee el 


13 0.7727 -0.4889 0 0.60 || 0.9166 0.4033 
EXERCISES 
6.1. Obtain AB, when 
A=) -1 0 0 and B=(|1 Ol. 
2 3 4 2 0 
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6.2. Compute 4°, 4? and 4*, when 


a-[) 2 
“13 4|' 


6.3. Form AB# and BA: 


Z 3 2 —| 2 1 
A=/1 0 0}; and B=] 2 3 4 |. 
Z 0 2 1 -2 3 


6.4. Compute 4~' and check your result by direct multiplication with A, 


where 
l Z 
a 3 A 


6.5. Find the inverse in the following cases: 


2 4 3 1 6 4 
(a) | 0 l l (b) |}0 2 3). 
2 2 -1 0 l 2 
6.6. Compute the inverse of the matrix 
3 2 4 
2 ] 1 
I 3 § 
and use the result to solve the system of equations: 
3x +2y4+42=7 
2x+yper=7 
x+3y +52 =2., 
6.7. Find whether the following systems are consistent, or not: 
(a) 2x-y+z=4 (b) 5x-3y+7z=4 
3x—y+z=6 3x + 26y+22=9 
4x—y+2z=7 7x+2y+l0z=5. 
—x+y-2=9 


6.8. Show that the equations 
X, +2xy —X3 =3 
X —X_ + 2x; =1 
2x, — 2x, +3x; =2 
x, —X_ +X, =-1 
are consistent and solve them. 
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6.10. 
6.11. 


6.1 2. 


6.13. 


6.14. 


6.15. 


6.16. 
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Use Gaussian elimination with partial pivoting to solve the system 
2x, + Xq —Xy = —1 
x, — 2x, +3x3 =9 
3x, — x2 + 5x; =14. 
Check your answer by substituting into the original equations. 
Use Gauss—Jordan method to solve the system in Problem 9. 
Find the inverse of the matrix: 
1 al | 1 
A=/1 —2 4 


l 2 2 
using Gauss elimination. 
Solve the system 
5x-2y+z=4 
Ix+ y-53z7=8 
3x+7y+4z=10 


by (a) Gauss elimination (b) Gauss—Jordan method. In both the cases, 
check your answers by substituting them into the original equations. 


Decompose the matrix 


5 
A=|7 | —§ 
3 7 4 


into the form ZU and hence solve the system Ax=b where 
b=[4 8 10]'. Determine also Z~' and U~' and hence find 4-'. 
Develop a subprogram in a language of your choice to solve a system 
of equations using Gauss elimination with partial pivoting. Test 
your subprogram using the system given in Problem 12. 
Develop a subprogram in a language of your Choice to decompose 
a matrix A into the form LU using partial pivoting. Include the 
possibility of computing the inverse also. Test your program with 
the matrix given in Problem 13. 
Solve the system of equations: 
2x-y=0 
—x+2y-z=0 
-y¥+2z-u=0 
-z+2u=1 
by the procedure described in Section -6.3.8. 
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(a) fl 6 1 al “a (cy) [ -9 2 6 


1 2 0 V2 1 5 oO -31. 
0 0 3 -16 4 11 
6.23. Determine the largest eigenvalue and the corresponding eigenvector 
of the matrices 
(a) |10 -2 \ (bo) [1 3 =1 
—2 lo 20-2 3 2 4). 
l ~2 10 =1 4 10 


6.24, Use the iterative method to find the largest eigenvalue and the 
corresponding eigenvector of the matrix 


5 2 | =2 


4"); 3 19) (2 
-2 —4 2 l 
6.25. Reduce the following matrices to the tridiagonal form by Householder’s 
method 
(a) fl 3 4 0) [2 -1 -l 
3 I 2 -| 2 ii. 
4 2 1 -] -i 2 
6.26. Use the OR algorithm to find the eigenvalues of the matrix 
0 1 4 
A=|1 3) OD. 
‘ia 0 
6.27. Compute the SVD of the matrix 
l 2 
A=|2 Ii. 
1 3 


6.28. Find the eigenvalues and the corresponding orthonormalized eigen- 
vectors of the matrix A'A if 


20 4 
A=|10 14 
5 5 


Hence determine the SVD of A. 
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Numerical Solution of Ordinary 
Differential Equations 


7.1 INTRODUCTION 


Many problems in science and engineering can be reduced to the problem 
of solving differential equations satisfying certain given conditions. The 
analytical methods of solution, with which the reader is assumed to be 
familiar, can be applied to solve only a selected class of differential equations. 
Those equations which govern physical systems do not possess, in general 
closed-form solutions, and hence recourse must be made to numerical 
methods for solving such differential equations. 

To describe various numerical methods for the solution of ordinary 
differential equations, we consider the general first order differential equation 


@ = f(x,y) (7.12) 
with the initial condition, 
y(%) = Fo (7.1b) 


and illustrate the theory with respect to this equation. The methods so 
developed can, in general, be applied to the solution of systems of first- 
order equations, and will yield the solution in one of the two forms: 


(i) A series for y in terms of powers of x, from which the value of 
y can be obtained by direct substitution. 


(ii) A set of tabulated values of x and y. 


The methods of Taylor and Picard belong to class (i), whereas those 
of Euler, Runge-Kutta, Adams—Bashforth, etc., belong to class (ii). These 


295 
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latter methods are called step-by-step methods or marching methods because 
the values of y are computed by short steps ahead for equal intervals A of 
the independent variable. In the methods of Euler and Runge—Kutta, the 
interval length & should be kept small and hence these methods can be 
applied for tabulating y over a limited range only. If, however, the function 
values are desired over a wider range, the methods due to Adams—Bashforth, 
Adams—Moulton, Milne, etc., may be used. These methods use finite-differences 
and require ‘starting values’ which are usually obtained by Taylor’s series 
or Runge-Kutta methods. 

It is well-known that a differential equation of the mth order will have 
n arbitrary constants in its general solution. In order to compute the numerical 
solution of such an equation, we therefore need mn conditions. Problems in 
which all the initial conditions are specified at the initial point only are 
called initial value problems. For example, the problem defined by Eqs. (7.1) 
is an initial value problem, On the other hand, in problems involving second- 
and higher-order differential equations, we may prescribe the conditions at 
two or more points. Such problems are called boundary value problems. 

We shall first describe methods for solving initial value problems of the 
type (7.1), and at the end of the chapter we will outline methods for solving 
boundary value problems for second-order differential equations. 


7.2 SOLUTION BY TAYLOR'S SERIES 


We consider the differential equation 

y =f y) (7.1a) 
with the initial condition 

¥(Xq) = Yo- (7.1b) 
If (x) is the exact solution of (7.1), then the Taylor’s series for y(x) around 
x =x, is given by 


= Z 
y(x) = yo + (x- x9) Sa yi 4 (7.2) 


If the values of yp, yp,--. are known, then (7.2) gives a power series for y. 
Using the formula for total derivatives, we can write 


yr=f'=ftyfh = htt, 
where the suffixes denote partial derivatives with respect to the variable 
concerned. Similarly, we obtain 


y= I" = feet Spf + £ Lyx + fy f+ Hye thf) 


= fa +2 + F fy + Subp + Hy 


and other higher derivatives of y. The method can easily be extended to 
simultaneous and higher-order differential equations. 
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Example 7.1 From the Taylor series for p(x), find (0.1) correct to four 
decimal places if y(x) satisfies 


yi=x-y? and y(0)=1. 


The Taylor series for p(x) is given by 
2 


v(x) =1+ xy + aye ype Zayl zy! oe 
2 6°° 24 120 


The derivatives yp, yg.... ete. are obtained thus: 


yQ@=x-y yo =— 
y"(x) =1-2py" yy =3 
y"(x) = -2yy” -2y" Yy=—t 
y (x) = -2yy" —6y/y" yg =34 
py” (x) =—2yy™ -By'y'"-6y"" yg =—186 


Using these values, the Taylor series becomes 

Va _3l is of aa 

12 

To obtain the value of (0.1) correct to four decimal places, it is found that 

the terms up to x* should be considered, and we have »(0.1)=0.9138. 
Suppose that we wish to find the range of values of x for which the 

above series, truncated after the term containing x‘, can be used to compute 

the values of » correct to four decimal places. We need only to write 


- <0.00005 or x <0.126. 


y(x)elnxese -304 


Example 7.2 Given the differential equation 

yay Hye 
with the conditions y(0)=1 and »'(0)=0, use Taylor's series method to 
determine the value of y(0.1). 


We have y(x)=1 and y'(x)=0 when x=0. The given differential 
equation 15 


y"(x) = xy'(x) + ¥(x) (i) 
Hence y'(0)= y(0)=1. Successive differentiation of (1) gives 
yx) = p"(x) + v(x) + ye) = e"(*) + 2y'Q), (ii) 
yl" (x) = xy""(x) + y"(x) + 2y"(x) = 29") + 3"), (iii) 
y" (x) = xy" (x) + p(x) + 3") = ay" (x) + 4"), (iv) 
y"'(x) = xy" (x) + (x) + dy (x) = ay" (2) + Sp" (0), (v) 
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and similarly for higher derivatives. Putting x=0 in (ii) to (v)}, we obtain 


y"(0)=2y'(0)=0, yy" (O)=3y"(0)=3, yy" (O)=0,  y"(O)=5. 
By Taylor's series, we have 


_ x? nr x ne - iv 
x)= 90) + WO) + SY O}+—¥ () +55 (0) 
6 


5 

x x i 

— y" (0)+— yp” os 
“720” 90” we 


Hence 


OME OW AOD. ee. 
y(0.1)=14+-— +" -G) +6) + 


=1+ 0,005 + 0,0000125, neglecting the last term 


= 1,0050125, correct to seven decimal places. 


7.3 PICARD’S METHOD OF SUCCESSIVE APPROXIMATIONS 


Integrating the differential equation in (7.1), we obtain 


y=yo+ | fy)ae. (7.3) 

ei] 
Equation (7.3), in which the unknown function » appears under the integral 
sign, is called an integral equation. Such an equation can be solved by the 


method of successive approximations in which the first approximation to y 
is obtained by putting yp for y on right side of (7.3), and we write 


£ 
Paine | femsrde 
ay 


The integral on the right can now be solved and the resulting »"” is substituted 
for y in the integrand of (7.3) to obtain the second approximation y”’: 


= 
YP ayy f Sx, 0c 
% 
Proceeding in this way, we obtain py), yl, pr) and yl), where 


x 
yi”) = Vp + j f(x, yh) de with y = Vo (7.4) 


ai 
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If we assume the continuity of df/dy, then the expression in the brackets 
can be simplified by using the mean value theorem [see Theorem 1.5, 
Chapter 1] 


Ff (Xn» Yn) — F Ons Yn) = (Yn — VOI Fy Cons Sn) = Cn Fy Ons Suds 


where ¢, lies between y(x,) and y,. We thus have obtained the recurrence 
formula 


Cnt] = en + he, f, (Xs bn) t+ Ray + Laat 
= €n(1+ Af, Ons Sul + Raat + Last (7.12) 


The first term on the right side of (7.12) is the propagated error, i.e. the 
error in y,,4, resulting from the error in the previous approximation y,. 
Expressions for é,., can be obtained by successive substitutions into 
(7,12). Thus we obtain 
& =0, 
eq =h +h, 
ey =[1+ Af, Hs oR +4) +R +L, 


és =[1 + Af, (a, Sn) ML + Af Cy, 61) CR +) +R +1n}+ Ry +L, 


and so on. 

An upper bound for the total solution error can be obtained analytically, 
but this will not be attempted here. The interested reader is referred to the 
book by Isaacson and Keller for more details. The step-by-step calculation 
of the solution error using relation (7.12) is demonstrated in the following 
illustrative example. 


Example 7.6 We consider, again, the differential equation y’=—-y with 
the condition »(0)=1, which we have solved by Euler’s method in Example 7.5. 


Choosing 4=0.01, we have 
1+ Afy(Xqs Eq) =140.01(-1) = 0.99. 
and 
Lyon =H y"(0,) = 0.00005 (pq) 


In this problem, p(,,)< )(x,), since y'’ is negative. Hence we successively 
obtain 


[4] $0.00005=5x107, 
|L>| < (0.00005) (0.99) <5x 107, 
|Lq| $(0.00005) (0.9801) <5«107, 
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and so on. For computing the total solution error, we need an estimate of 
the rounding error. If we neglect the rounding error, i.e. if we set 


Raat =, 


then using the above bounds, we obtain from (7.12) the estimates 
ey = 0, 
le |s5x107 
ley] <0.99e, +5x107> <107 
je3]<0,99e, +5x10° <107 +5x107 
|e4] 50.992, +5x 107° <107 +107 =2x10 = 0.0002 


It can be verified that the estimate for e, agrees with the actual error in the 
value of (0.04) obtained in Example 7.5. 


7.4.2. Modified Euler's Method 


Instead of approximating f(x, y) by (xg, 9) in (7.6), we now approximate 
the integral in (7.6) by means of trapezoidal rule to obtain 


i = Yo + ELL C0. Yo) + Fla] (7.13) 


We thus obtain the iteration formula 


yf") = yy +2 Lf (x, Yo) + Fx. ¥f"), nn =0,1,2,... (7.14) 


where y}") is the mth approximation to ;. The iteration formula (7.14) can 
be started by choosing yl ) from Euler's formula: 


y{” = 9 + Hf (Xp. Yo). 
Example 7.7 Determine the value of y when x=9.1 given that 
y(OQ)=1 and y'a=x*+y 


We take 4=0,.05, With x,=0 and yp =1.0, we have f(x», ¥9)=1.0. Hence 
Euler's formula gives 


y() =140,05(1) =1.05 


Further, ae =0.05 and f(x, ¥ (Oh) =] 0525. The average of f(xj. ¥9) and 
F (4, y\? Y) is 1.0262. The ie of yl! ) can therefore be computed by using 
(7.14) and we obtain 


yf 21.0513, 
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Repeating the procedure, we obtain y(?) =1.0513, Hence we take y, =1.0513, 
which is correct to four decimal places. 

Next, with x, = 0.05, », =1.0513 and 4=0.05, we continue the procedure 
to obtain 5, i.e. the value of y when x=0.1. The results are 


yf =1.1040, yf =1.1055,  y =1.1055. 
Hence we conclude that the value of y when x=0.1 is 1.1055. 


7.5 RUNGE-KUTTA METHODS 


As already mentioned, Euler’s method is less efficient in practical problems 
since it requires A to be small for obtaining reasonable accuracy. The Runge— 
Kutta methods are designed to give greater accuracy and they possess the 
advantage of requiring only the function values at some selected points on 
the subinterval. 


If we substitute y, = yp + Af (xg, Yo) om the right side of Eq. (7.13), we 
obtain 
h 
N= Yo +5 Ufo + Fl +A, Yo + Mod, 


where /p = (xq, ¥o). If we now set 


k=hfyo and ky =hf(%y +h, yo +h) 
then the above equation becomes 


yi =o +5(h +k), (7.15) 


which is the second-order Runge-Kutta formula. The error in this formula 
can be shown to be of order 4’ by expanding both sides by Taylor’s series. 
Thus, the left side gives 


Fer 


he i 
Yo + hy + w+ yor 
and on the right side 


b= Gotha hir=h| fondo L sour] 
Since 
df(x.y) F Ff 
ay ed : 
we obtain 


ky =hLfy + hfj + Oh?) =hfy +? h+O0P), 
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where the parameters have to be determined by expanding both sides of the 
first equation of (7.18a) by Taylor's series and securing agreement of terms 
up to and including those containing A*. The choice of the parameters is, 


again, arbitrary and we have therefore several fourth-order Runge-Kutta 
formulae. If, for example, we set 


ay = fy ==, a=) a@, =1, 


(7.19) 
l I 
vy aia: i ¥2 SO? a ase 
W, =W, =! w -i{1-5) Ww. =i {14 
17 "6 2°3 J2 ] 3 V2 ? 
we obtain the method of Gill, whereas the choice 
saat Rays 
ay = 1-3? 1=5 
W =W,=-, HW, =, == 


leads to the fourth-order Runge-Kutta formula, the most commonly used 
one in practice: 


Y= Yo +2(hy + 2ky + hy + hy) (7.21a) 
where 


ky =hf'(xo, Yo) 


k = i{x thaw 4th | 
(7.21b) 
ky = i( +h ‘i 3h, | 


Kg = hf (xq +h, yo +) 


in which the error is of order 4°. Complete derivation of the formula is 
exceedingly complicated, and the interested reader is referred to the book 


by Levy and Baggot. We illustrate here the use of the fourth-order formula 
by means of examples. 
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We take h=0.2. With x, = yy = 0, we obtain from (7.21a) and (7.21b), 
k, = 0.2, 
ky = 0.2 (1.01) = 0.202, 
k, = 0,2 (1+ 0.010201) = 0.20204, 
k, = 0.2 (1 + 0.040820) = 0.20816, 


y(0.2)=04 : (k, + 2ky + 2k; +k,) = 0.2027, 


which is correct to four decimal places. 


To compute y(0.4), we take xy =0.2, yp =0.2027 and fA =0,2, With 
these values (7.21a) and (7.21b) gives 


k, = 0.2 [1 + (0.2027)* ] = 0.2082, 
ky =0.2[1 + (0.3068) ] = 0.2188, 
k, =0.2[1 +(0.3121)7] = 0.2195, 
kg =0.2 [1+ (0.4222) ] = 0.2356, 


and 
y (0.4) = 0.2027 + 0.2201 = 0.4228, 


correct to four decimal places. 


Finally, taking xp =0.4, yp =0.4228 and h=0.2, and proceeding as 
above, we obtain y(0.6)=0,6841, 


Example 7.10 We consider the initial value problem y'=3x+ y/2 with the 
condition y(0)=1. 


The following table gives the values of 1(0.2) by different methods, the 
exact value being 1.16722193. It is seen that the fourth-order Runge—Kutta 
method gives the accurate value for h=0,05. 


Mathod h Computed value 
Euler 02 1.100 000 00 
0.1 1.132 500 00 
0.05 1.149 567 56 
Modified Euler 0.2 1.100 000 00 
0.1 1.150 000 00 
0.05 1.162 862 42 
Fourth-order Runge-Kutta 0.2 1.167 220 83 
0.1 1.167 221 B6 
0.05 1.167 221 93 
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7.6 PREDICTOR-—CORRECTOR METHODS 


In the methods described so far, to solve a differential equation over a single 
interval, say from x=x, to x=x,,), we required information only at the 
beginning of the interval, i.e. at x =x,. Predictor—-corrector methods are the 
ones which require function values at x,,x,_),X,-3,--. for the computation 
of the function value at x,,,.A predictor formula is used to predict the 
value of y at x,,, and then a corrector formula is used to improve the value 
of Vn+i- 

In Section 7.6.1 we derive Predictor—corrector formulae which use 
backward differences and in Section 7.6.2 we describe Milne’s method 
which uses forward differences. 


7.6.1 Adams—Moulton Method 
Newton's backward difference interpolation formula can be written as 


ey ee “Ca2) 


we 
fot 6 


F(%, Y= fg + HV fo + 


n(a+l) 
2 


where 


n=—— and = fy = f (%o. Yo) 


If this formula is substituted in 


W=M+ | F(x, y)dx, (7.23) 
<q 
we get 

W=yo+ + fa snng Mey fo +a 
xy 
| | 

oe les 
] 

z Pr ee gee 

wen(ledve sy ie 720" tl 


It can be seen that the right side of the above relation depends only on yp, 
y_y. Pea...) all of which are known, Hence this formula can be used to 
compute y,. We therefore write it as 


3 251 
P 1+—-V4+—V¥' 4-7 +9 -) 7.24 


Presented By: http://www.ebooksuit.com 


310 CwaptTer 7: Numerical Solution of Ordinary Differential Equations 


This is called Adams—Bashforth formula and is used as a predictor formula 
(the superscript p indicating that it is a predicted value). 

A corrector formula can be derived in a similar manner by using Newton's 
backward difference formula at /): 


fe y= fi tnVf ae a] ROD je. 25) 
Substituting (7.25) in (7.23), we obtain 


a(n+]) 


n=yo+ | [i +nng 


Vf | 
% 


0 
yok fj snig «22D ath iu 
1 


l eae ee 195 +.) 
= yp +h| 1-—V-—V¥? -— V3 - 7.26 
** 2°32” “24” 70% 4 Me 
The right side of (7.26) depends on ,, ¥), ¥.),... Where for y, we use y?, 
the predicted value obtained from (7.24). The new value of y, thus obtained 
from (7.26) is called the corrected value, and hence we rewrite the formula as 
yiay th(1-3v-tv ty By 


12 24 720° oy ey) 


This is called 4dams—Moulton corrector formula the superscript ¢ indicates 
that the value obtained is the corrected value and the superscript p on the 
right indicates that the predicted value of », should be used for computing 
the value of f(x,, ))- 

In practice, however, it will be convenient to use formulae (7.24) and 
(7.27) by ignoring the higher-order differences and expressing the lower- 
order differences in terms of function values. Thus, by neglecting the fourth 
and higher-order differences, formulae (7.24) and (7.27) can be written as 


yt =o 2 (35,fg -59F_ +37 f.3 -9f.;) (7.28) 


h 
Yi =o +52 OFP +19f-Sf4 + Fa) (7.29) 
in which the errors are approximately 


i ty (4) and _ aa fo (4) respectively. 
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h 
Foal = Vn-] +5 Unt +4fo+ fay) (7.34a) 


The application of this method is illustrated by the following example. 


Example 7.12 We consider again the differential equation discussed in 
Examples 7.9 and 7.10, viz., to solve p'=1+* with y(0)=0 and we wish 
to compute }(0.8) and »(1.0). 

With 4=0.2, the values of ~(0.2), (0.4) and (0.6) are computed in 
Example 7.9 and these are given in the table below: 


x y yet+y® 
4] 0 1,0 

o2 0.2027 1.0417 
0.4 0.4228 1.1787 


0.6 0.6841 1.4681 
To obtain (0.8), we use (7.32) and obtain 


¥(0.8) = 0+ = (2 (1.041 1) — 1.1787 + 2(1.4681)] = 1.0239 


This gives 
y'(0.8) = 2.0480. 


To correct this value of »(0.8), we use formula (7.34) and obtain 
(0.8) = 0.4228-+ 9 [1.1787 +4(1 A681) + 2.0480] = 1.0294, 


Proceeding similarly, we obtain y(1.0)=1.5549, The accuracy in the values 
of y(0.8) and »(1.0) can, of course, be improved by repeatedly using formula 
(7.34). 


Example 7.13 The differential equation y’=x* + y* -2 satisfies the following 


data: 
x ¥ 
0.1 1.0900 
0 1.0000 
0.1 0.8900 
02 0.7605 


Use Milne’s method to obtain the value of »(0.3). 
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We first form the following table: 


x y y=x@+y?-2 
-0.1 1,0900 0.80190 

0 1.0 -1.0 

o4 0.8900 -1.19790 

02 0.7605 ~1,38164 


Using (7.32), we obtain 
y(03)= 1.09 +S [2¢-1)- (1.19790) + 2(-1.38164)] = 0.614616. 
In order to apply (7.34), we need to compute y’(0.3). We have 
y'(0.3) = (0.3)? +(0.614616)* - 2 = -1.532247. 
Now, (7.34) gives the corrected value of (0.3): 


y(0.3) = 0.89 + = [-1.197900 + 4(-1.38164) + (-1.532247)] = 0.614776. 


7.7 THE CUBIC SPLINE METHOD 


The governing equations of a cubic spline have been discussed in detail in 
Section 3.14, where the cubic spline function has been obtained in terms of 
its second derivatives, M4,. In certain applications, e.g. the solution of initial- 
value problems, it would be convenient to use the governing equations in 
terms of its first derivatives, i.e. m, Using Hermite’s interpolation formula 
(see Section 3.9.3), it would not be difficult to derive the following formula for 
the cubic spline s(x) in x;_, SxS x, in terms of its first derivatives »'(x;)= m,: 


s(x) = (x - re ~ xy ) -m, (x- sul (x, -Xx) 


(x, — x)? (2(x—x,_,) +4] o (x- x4) a —*)+4l (7.35) 


where A = x, —x,_,. Differentiating (7.35) with respect to x and simplifying, 
we obtain 


+ ¥j-4 


s'(x)= me ~ x) (2x,_, + x; -3x) a (x —x)_1) (X41 + 2x, -3x) 
6(y; — ¥i-1) 


+ iB 


(x —x)_;)(x; — x). (7.36) 


Again, 


s"(x)=- ars (x)_) + 2x, —3x)- Hx. +x, — 3x) 
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aaa me + x, — 2x), (7.37) 
which gives 
i 2m,_, 4m, 6 
5 Game ~ Ya) 
2m; 4m, 6 
ae ae “ae ho — 5,4). (7.38) 
If we now consider the initial-value problem 
ay 
—= f(x, 7.39 
ie (7.39a) 
and 
v(x) = ¥% (7.39b) 
then from (7.39a), we obtain 
dy _F Fb 
dette Gy de 


or 
¥"(4) = AO ¥)+ HO kK) Fp HY) 
= fi 5) + f(x. 5;) FO 8). (7.40) 
Equating (7.38) and (7.40), we obtain 


at +S —8,.1)" fy 5))+ f(y, 5) F(X: 5;) (7.41) 


from which s; can be computed. Substitution in (7.35) gives the required 
solution. 


The following example demonstrates the usefulness of the spline method. 
Example 7.14 We consider again the boundary-value problem defined by 
waa +5 y(0)=1, (a 

whose exact solution is given by 


y =13e"*-6x-12 (ii) 


We take, for simplicity, m=2, ie. 4 = 0.5 and compute the value of (0.5). 
Here f(x, y)=3x+/2 and therefore we have f/ =3 and fj =1/2. Also, 


1 
F(x, 5;) = 3x, rate 
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In a similar manner, one can extend the Taylor series method or Picard’s 
method to the system (7.42). The extension of the Runge-Kutta method to 
a system of nm equations is quite straightforward. 

We now consider the second-order differential equation 


y =Fi(xy¥) (7.44a) 
with the initial conditions 


¥(%q)= Yo and y'(xq) = yo. (7.45a) 
By setting z=’, the problem (7.44a) and (7.45a) can be reduced to the 
problem of solving the system 


yo=z and 2 =F(x, y, 2) (7.44b) 
with the conditions 


y(%p)=¥o and § 2(xqy) =H (7.45b) 


which can be solved by the method described above. Similarly, any higher- 
order differential equation, in which we can solve for the highest derivative, 
can be reduced to a system of first-order differential equations. 


7.9 SOME GENERAL REMARKS 


In the preceding sections, we have given a brief discussion of some well- 
known methods for the numerical solution of an ordinary differential equation 
satisfving certain given initial conditions. If the solution is required over a 
wider range, it is important to get the starting values as accurately as 
possible by one of the methods described. 

It is outside the scope of this book to present a comprehensive review 
of the different methods described in this text for the numerical solution of 
differential equations, but the following points are relevant to the methods 
discussed. 

The Taylor’s series method suffers from the serious disadvantage that 
all the higher derivatives of f(x, y) (see Eqs. 7.1) must exist and that h 
should be small such that successive terms of the series diminish quite 
rapidly. Likewise, in the modified Euler method, the value of 4 should be so 
small that one or two applications of the iteration formula (7.14) will give 
the final result for that value of 4. The Picard method has probably little 
practical value because of the difficulty in performing the successive 
integrations. 

Although laborious, the Runge-Kutta method is the most widely used 
one since it gives reliable starting values and is particularly suitable when the 
computation of higher derivatives is complicated, When the starting values 
have been found, the computations for the rest of the interval can be 
continued by means of the predictor-corrector methods. 
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The cubic spline method is a one-step method and at the same time a 
global one. The step-size can be changed during computations and, under 
certain conditions, gives O(A*) convergence. The method can also be extended 
to systems of ordinary differential equations. 


7.10 BOUNDARY-VALUE PROBLEMS 


Some simple examples of two-point linear boundary-value problems are: 


(a) y'(x) + f(x)y¥' (x) + a(X)y (x) = r(x) (7.46) 
with the boundary conditions 

y(Q%)=a and y(x,)=6 (7.47) 

(b) y" (x) = p(x) y(x) = 9(x) (7.48) 

with 

y(%) = yi(xj)=A and y(x,)= y'(x,) = 8B. (7.49) 


Problems of the type (b), which involve the fourth-order differential equation, 
are much involved and will not be discussed here. There exist many methods of 
solving second-order boundary-value problems of type (a). Of these, the finite 
difference method is a popular one and will be described in Section 7.10.1. 
An alternate method, called the shooting method, will be described next. 
Finally, in Section 7.10.3, we, discuss a method based on the application of 
cubic splines. 


7.10.1 Finite-difference Method 


The finite-difference method for the solution of a two-point boundary value 
problem consists in replacing the derivatives occurring in the differential 
equation (and in the boundary conditions as well) by means of their finite- 
difference approximations and then solving the resulting linear system of 
equations by a standard procedure. 

To obtain the appropriate finite-difference approximations to the derivatives, 
we proceed as follows. 

Expanding y(x+/) in Taylor’s series, we have 


y(x +h) = yx) + hy'(x) + us y"(x)+ e p'(x)+e (7.50) 
from which we obtain 
yi(x)= POL Sy" 
Thus we have 
y(x)= Mee +O(A) (7.51) 
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We have explained the method with simple boundary conditions (7.47) 
where the function values on the boundary are prescribed. In many applied 
problems, however, derivative boundary conditions may be prescribed, and 
this requires a modification of the procedures described above. The following 
examples illustrate the application of the finitte-difference method. 


Example 7.15 A boundary-value problem is defined by 
y"+ytl=0, O<x<I 
where 
y(0)=0 and y(lj=0. 
With 4 =0.5, use the finite-difference method to determine the value of (0.5). 
This example was considered by Bickley [1968]. Its exact solution is 
given by 


y(x)=cosx+ 


cos! . 
1 sinx —l, 
from which, we obtain 

y (0.5) = 0.139493927. 


Here nh =1. The differential equation is approximated as 


Yt-29i* Yet gy 4120 
he 
and this gives after simplification 
Y4- (2-1) + Yi =-W, f= 2, ok 


which together with the boundary conditions », =0 and y, =0, comprises 
a system of (1+1) equations for the (n+1) unknowns yo, ¥),.--. Py. 
Choosing A=1/2 (ie. n=2), the above system becomes 
vi -(2-3] + aol 
0 4 WT 4° 
With YouRki = 0, this gives 


y= ¥(0.5)= ; = 0.142857142... 


Comparison with the exact solution given above shows that the error in the 
computed solution is 0.00336. 

On the other hand, if we choose #= 1/4 (i.e. n = 4), we obtain the three 
equations: 


31 
Yo Tg Mt 2 - "6 
31 1 
¥ 16. +; "Te 
31 l 
v2 1673 ¥a mats 
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It is possible to obtain a better approximation for the value of »(1.0) by 
extrapolation to the limit. For this we divide the interval [0,2] into two 
subintervals with h= 1.0. The difference equation at the single unknown 
point y, is given by 
Yo - 2 +2 = Mi 
Using the values of yy and ys, we obtain 
y, = 1.20895. 


Hence (7.61) gives 
4(1. 18428) = 1.20895 
yl oe 


which is a better approximation since the error is now reduced to 0.00086. 


= 1.17606, 


7.10.2 The Shooting Method 


This method consists in transforming the boundary value problem into an 
initial-value problem. Its main advantage is that it is easy to apply. The 
method requires good initial guesses for the first derivative and can be 
applied to both linear and nonlinear problems. To describe the method, we 
consider the boundary-value problem defined by 


¥'(x) = FO) y(0)=0, y=. (7.62) 
The main steps involved in this method are: 


(i) transformation of the boundary-value problem into an initial-value 
problem 


(ii) solution of the initial-value problem by any of the known methods, 
and finally 


(11) solution of the given boundary-value problem. 


To apply any initial value method, we must know y'(0). Let the true 
value of »'(0) be m. We start with two initial guesses for m and then 
determine the corresponding values of )(1) using any initial value method, 
Let the two guesses for m be m, and m, and also let the corresponding 
values of (1) be Yp and Yj, obtained by the initial value method. Using 
linear interpolation, we can then obtain a better approximation m, for m. 
This is given by 


ly — My 7 ity — My (7.63) 
yI)-%y 1 -%p 
which gives 
Mz = My + (my Aig at (7.64) 
h-% 
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With this value of m,, we solve the initial value problem 
y"(x) =f); y(O)=0, yO) =m, (7.63) 


and obtain F;. If this agrees with }(1) to the desired accuracy, the solution 
to the boundary-value problem is obtained, Otherwise, linear interpolation is 
carried out with (m,, ¥,) and (m, ¥,) to obtain m;. The process is repeated 
until convergence is obtained, i.e. until the value of F,; agrees with (1) to 
the desired accuracy. The speed of convergence depends on how good the 
initial guesses were. The method will be tedious to apply to higher-order 
boundary-value problems and in the case of nonlinear problems, linear 
interpolation yields unsatisfactory results. 

The method 1s illustrated with a simple linear second-order boundary- 
value problem. 


Example 7.17 Solve the boundary value problem 
y'(xy=y); yO) =0, yl) =1.1752. 


by the shooting method, taking my =0.7 and m, =0.8. By Taylor’s series, 
we have 

x? i x ff e4 
yx) = ¥O)+9'0) + O+¥ (0) +" (0) 


x , 
+702 “(+ y" (OO) +++ (i) 


Since p"(x)= y(x), we have 
yay x) y"@=y"(x)= y@), 
y*@)=y@),  y"@) =r") =v)... 
Putting x=Qin the above, we obtain 
y'(0)= y()=0,  y'"(0)= y'(0), 


y'"(0)=0, y" (0) = y'(0),... 
Substitution in (i) gives 
rer yx x 
— +——+—_ + —_ } -.. 0)=0. 
y(x)= vol r++ +750 * Soao * 365800 7 : since y(0) 


Hence 


y= yO(I+t +e 


1 ‘ 
a age ro] y'(0) (1.1752) (ii) 


With y'(0) = mp =0.7, (ii) gives 
y(l) = ¥, = 0.8226. 
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If we divide the interval [0, 1] into two equal subintervals, then from 
Eq. (7.62) and the recurrence relations for Af, we obtain 


3 : 
5) =— = 0.13636, i 
y (0.5) 63 (ii) 
Then 
25 
My=-l, M,=-—-, M,=-1 


Hence we obtain 
47 47 
(0) =—, (WQ=-—, (0.5)=0. 
s (0) a8 (1) rT s'(0.5)=0 


From the analytical solution of the problem (i), we observe that »(0.5) 
= 0.13949 and hence the cubic spline solution of the boundary-value of the 
problem has an error of 2.24% (see Bickley [1968]). 


Example 7.19 Given the boundary-value problem 
xy"+ay'-y=0; yO)=L  y(2)=0.5 
apply the cubic spline method to determine the value of )(1.5). 


The given differential equation is 


1, 1 : 
yh =——y'+—y. (i) 
ad x 


Setting x =x, and y"(x,)=M,, eq. (i) becomes 


M, a—— yl tay yy, (ii) 
*j x; 


Using the expressions in (7.67) and (7.68), we obtain 


u,=-2 (4a, BM +22) Fy, i=, 1, 2,....0-1. Gi 
a aj 


3 6 A 


M=-1 (bag + Ba rote Ty, ie Quunm Gv) 
3 


If we divide [1,2] into two subintervals, we have A= 1/2 and n= 2. Then 
eqs. (iii) and (iv) give 
10My —-M, +24y, =36 


16M, ~My, -32);, =-12 
My + 20M, +16), = 24 
M; + 26M; —24y, =-9 
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With h= 1/2, we obtain 


Yo + 4y, + = 24 (9 -— 27, +72) 
Since y, =1, the above equation becomes 


Yo +4, = 24 (yg — 2) +23 
or, equivalently 


S2y =23 yp + 23 (1x) 
- For the derivative boundary condition, we use Eq. (7.68) and obtain 
Yo =0=-2M, - iM, +2(4 - Yo) 
Since Af, = yp and M,=),, the above equation gives 
2y¥p + = 24 (4 — Yo) (x) 
Equations (ix) and (x) yield 


598 
= y(0.5) = — =0.7266. 
yy = ¥(0.5) 3 


Thus the error in the cubic spline solution is 0.0044, This example demonstrates 
the superiority of the cubic spline method over the finite difference method 
when the boundary value problem contains derivative boundary conditions. 


EXERCISES 


7.1. Given 


Olax and y(0)=1, 


obtain the Taylor series for »(x) and compute »(0.1) correct to four 
decimal places. 
7.2. Using the Taylor’s series method, prove that the solution of 


Zz 
TY 4 y=0 
ax 
is given by 


x 6COF 


when the conditions are x=0, y=d and dy/dx =0. 
73. if 


yoa[i-E ies ‘Heat ,..} 


dy zz u 

de x*+y ‘ 
where y(4)=4, compute the values of y(4.1) and »(4.2) by Taylor's 
series method. 


Presented By: http://www.ebooksuit.com 


Hidden page 


Presented By: http://www.ebooksuit.com 


330 Cuapter 7: Numerical Solution of Ordinary Differential Equations 


7.13. 


7.14, 


7.15. 


7.16, 


7.17. 


7.18. 


Given the problem 


@ = flxy) and y(%y)= yo 


an approximate value y, at x= x, is given by the formula 
w=% +h + 4k, + ky)+ Ry, 
where 


k, = hf (x9. Yo), 


I 1 
ky = =f, —k 
2 It (x05 Yo ; 
ky = hf(xp +h, yo + 2k2 - kj). 
Show that R, is of order A’. 
Tabulate the solution of the equation 


x20)", y(Q)=0 


for the range 0<x<0.5 at intervals of 0.1. Obtain the solution 
correct to four decimal places, and compare it with the Taylor’s 
series solution. 


Use the Runge-Kutta method to solve 
dy 2 
10— =x" +’, 0)=1 
+y y(0) 


for the interval 0<x<0.4 with h=0.1. 

Use the predictor—corrector formulae for tabulating a solution of 
dy 2,2 

l — 1 = 
0 Pie deed y(Q)=1 
for the range 0.5.x 31.0. 

Tabulate the solution of 


Berry, y(Q)=0 


for 0.4<x<1.0 with 4=0.1 using the predictor—corrector formulae. 
Using Milne’s method, find )(0.8) given that 


Bax-y*, yO=0, y(02)=0.02, 


y(0.4)=0.0795, (0.6) = 0.1762. 
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7.20. 


7.21. 


Pade 


7.23. 


7.24. 


7.25. 


7.26. 
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Using Milne’s method, solve the differential equation 
+) 2+ y=0, with »(0)=2, 
for x=1.5 to x =2,5, Obtain the starting values by using the fourth- 
order Runge-Kutta method with 4 =0.5. 
Solve the system of differential equations 


dy 
— -f, a ri 
dt ¥ at oe 


with x=1, y=! when ¢=0, taking At=A=0.1. 
Solve the differential equation 
2 2 
fy _(2) +y" = (}, 
dx* dx 


with y(0)=1 and y'(0)=0, using the fourth-order Runge-Kutta 
method with 4=0.2. 


Given that 
dv dw wv 
— = VW, ort Roe | 
alu duos W 
and 
l 
nl)= w(l)=1. 


compute v(1.1) and w(1.1). 
Solve the equation 
fy +y=0 
with the conditions y(0)=1 and y'(0)=0. Compute y(0.2) and 
y(0.4). 
Solve the boundary-value problem 
y"-64y+10=0; y(0)= y(l)=0 


by the finite-difference method. Compute the value of (0.5) and 
compare it with the true value. 


Use the spline method to solve the initial value problem 
xdy+2yde=0 and y(2)=1 


Using the cubic spline technique, solve the following boundary- 
value problems 


(i) y"-y=0, y(0)=0 and y(l)=1 
(ii) py" +2y'+y=30x, y(0)=0 and p(I)=0 
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V2. 


7.28. 


7.29, 


7.30. 


(iii) y"-64y4+10=0, »(0)= y(1)=0. 
In each case, divide the given interval into two equal subintervals, and 


compare your solution with the analytical solution at the midpoint 
of the interval. 


Solve the boundary value problem 

y"=(x); (0) = 0, y(=1, 
by the shooting method. 
Solve the boundary-value problem 


y'-64y+10=0; y(0)=y()=0 


by the shooting method. 
Solve the boundary-value problem 
d*y 4x dy 2 | 
ive ad te 
with the boundary conditions y(0)=1 and y(2)=0.2. Use the cubic 
spline method first with , =| and then with A=1/2 to determine the 
value of (1). Compare your answers with the exact value obtained 
from the analytical solution y =1/(1+ x”) [Albasiny and Hoskins]. 


Solve the boundary-value problem 


2 
(l+xy s+ ry B-y- 0, 
with 
y(O)=1 and y(I)=0.5. 


Use the cubic spline method to determine the value of (0.5) and 
compare it with that obtained from the exact solution y=L/(1+ x). 
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CHAPTER 


Numerical Solution of Partial 
Differential Equations — 


8.1 INTRODUCTION 


Partial differential equations occur in many branches of applied mathematics, 
for example, in hydrodynamics, elasticity, quantum mechanics and electro- 
magnetic theory. The analytical treatment of these equations is a rather 
involved process and requires application of advanced mathematical methods. 
On the other hand, it is generally easier to produce sufficiently approximate 
solutions by simple and efficient numerical methods. Several numerical methods 
have been proposed for the solution of partial differential equations, but only 
the finite-difference methods have become popular and are more gainfully 
employed than others. We will therefore restrict ourselves to a treatment of 
the finite-difference methods and, in the sequel, we will discuss, very briefly, 
some of the numerical procedures with simple illustrative examples. Only 
the rudiments of the method will be given here and the interested reader is 
referred to the text by G.D. Smith (see Bibliography) for further details. 

The general second-order linear partial differential equation is of the 
form 


which can be written as 
Ati, + Bug, +Cuyy + Du, + Eu, + Fu =G, (8.1) 


where 4, B, C,...,G, are all functions of x and y. 
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Equations of the form (8.1) can be classified with respect to the sign 
of the discriminant. 

A, = B* -4A4C, (8.2) 
in the following way. If A, <0 at a point in the (x, y) plane, the equation 
is said to be of elliptic type, to be of Ayperbolic type when A, > 0 at that 
point, and to be of parabolic type when A, =0. 

In the following, we will restrict ourselves to three simple particular 
cases of Eq. (8.1), namely 


U,, + uy, =O (the Laplace equation) (8.3) 
Wey 3 ut, =0 (the wave equation) (8.4) 
u,, —u, =0 (the heat conduction equation), (8.5) 


where (x, y) are space coordinates and ¢ is the time coordinate. It is easy 
to see that the Laplace equation is of elliptic type, that the wave equation 
is of hyperbolic type and that the heat equation is of parabolic type. 

In a similar way, we conclude that the partial differential equation 


XU, +u,y, =0 


(i) parabolic if x=0 
(ii) elliptic ifx>0 
(iii) hyperbolic if x< 0. 


It is clear that the region plays an important role in the classification of 
partial differential equations. In the study of partial differential equations, 
usually three types of problems arise: 

(i) Dirichlet's problem: Given a continuous function f on the boundary 
C of a region A, to find a function w satisfying the Laplace equation in 2, 
ie. to find w such that 


u =OinR 
a ty, =0 in ea 
u= fon Cc 
(ii) Cauchy's problem: 
uy —U,, =0 for t>0 
u(x, 0) = f(x) (8.7) 
u(x, 0) = g(x) 
J (x) and g(x) being arbitrary. 
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(iii) u,-u,, =0 aes, 
u(x, 0) = f(x) 


These problems are all well-defined (or well-posed) and it is proved in 
textbooks of partial differential equations that they possess unique solutions. 
At this juncture, it is, however, important to mention a point of difference 
between ordinary and partial differential equations. In contrast with ordinary 
differential equations, the form of a partial differential equation is always 
connected with a particular type of associated conditions. Thus, the problem 
of Laplace’s equation with Cauchy boundary conditions, viz., the problem 
defined by 


(8.8) 


u(x,0)= f(x) (8.9) 
uy (x, 0) = g(x) 
is an ill-posed problem. 


8.2 FINITE-DIFFERENCE APPROXIMATIONS TO DERIVATIVES 


Let the (x, y) plane be divided into a network of rectangles of sides Ar=h 
and Ay= by drawing the sets of lines 


x=ih, i=0,1, 2... 
y=jk, j=0,1, 2... 


The points of intersection of these families of lines are called mesh points, 
lattice points or grid points. Then, we have (see Section 7.10 of Chapter 7) 


mt Seana aa + O(h) (8.10) 
= 00H) (8.11) 
Mies Fs y og? 12 
ah + O(A") (8.12) 
and 
=-2 + 

ge ey 5 lL 4 OUP) (8.13) 

where 


uy, = u(ih, jk) = u(x, y) 
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Similarly, we have the approximations 


We jay ~My | 
iy a + O(k) (8.14) 
= + O(k) (8.15) 
Hy jot — i, j=l 3 
= ft ke 
7 +O(k") (8.16) 
and 
iy ».j —2u, ; + 
yy = = LS" Ok?) (8.17) 


We can now obtain the finite-difference analogues of partial differential 
equations by replacing the derivatives in any equation by their corresponding 
difference approximations given above. Thus, the Laplace equation in two 
dimensions, namely 


has its finite-difference analogue 


l l 
pe ith ~ au yp +4, s) oy Mya —2uj +4 y4)=0. (8.18) 


If h=k, this gives . 


l 
uy y = 4 Mens + uj y +; jel +; y-i)s (8.19) 
which shows that the value of uw at any point is the mean of its values at 
the four neighbouring points. This is called the standard five-point formula 
(see Fig. 8.la), and is written 
Mist, pM Fe, po TY, j-1 ~ aug =0 (8.20) 


By expanding the terms on the right side of (8.19) by Taylor’s series, it can 
be shown that 


1 
eh er a | —4u; ; =h* (u,, + Uy, ) ~ 5B tzxy +O(h*) 


1 
= EM oy + OCH") (8.21) 


Instead of formula (8.19), we may also use the formula 


l 
a = q tis jolt Mat, fat * Mio, fod + 4-1, po) (8.22) 
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l 
Ug =< (us +07 +g ts); My air + Uy +s + ity). 


When once all the u;,(i=1, 2, 3, ...,9) are computed, their accuracy can 
be improved by any of the iterative methods described below. 


8.3.1 Jacobi’s Method 
Let ui? denotes the nth iterative value of u, ;. An iterative procedure to 
solve (8.20) is 


l 
u ee =k ee TH i. jru we Fu ul (8.26) 


for the interior mesh points. This is called the point Jacobi method. 


8.3.2 Gauss-Seidel Method 


The method uses the latest iterative values available and scans the mesh 
points systematically from left to right along successive rows. The iterative 
formula is: 


l 
Wie tN Ret Rees BTR, (8.27) 


It can be shown that the Gauss-Seidel scheme converges twice as fast as 
the Jacobi scheme. This method is also referred to as Liebmann'’s method. 


8.3.3 Successive Over-relaxation (or SOR Method) 


Equation (8.27) can be written as 


(net) (n) , be (mel), | in) (ntt) , tn) (n) 
Ti Su jj tal ig tM iat, tM TEE jt ea 
=i, 2 
apt j 


which shows that (1/4)R; ; is the change in the value of uw; , for one Gauss— 
Seidel iteration. In the SOR method, a larger change than this is given 
to af, and the iteration formula is written as 


intl). in) , | 
ui pit Bric 


1 
=5o[u eo tale) tu +a) 1+0-a)uf) (8.28) 


The rate of convergence of (8.28) depends on the choice of @, which is 
called the accelerating factor and lies between | and 2. 
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u (9 =< (14140+0)=05; 


u{) = (1+1+0+0)=05. 


The iterations have been continued using the formula (8.26), and seven 
successive iterates are given below: 


Uy Up Us Ly 


0.1875 0.1875 0.4375 0.4375 

0.15625 0.15625 0.40625 0.40625 
0.14062 0.14062 0.39062 0.39062 
0.13281 0.13281 0.38281 0.38281 
0.12891 0.12891 0.37891 0.37891 
0.12695 0.12695 0.37695. 0.37695 
0.12598 0.12598 0.37598 0.37598 


(b) Gauss—Seidel method: Five successive iterates are given below: 
Uy ™ Uy Ly 
0.25 0.3125 0.5625 0.46875 
0.21875 0.17187 0.42187 0.39844 
0.14844 0.13672 0.38672 0.38086 
0.13086 0.12793 0.37783 0.37646 
0.12646 0.12573 0.37573 0.37537 


(c) SOR method: With w=1.1, three successive iterates obtained by 
using the formula (8.28) are given below. 
us Ue Us Ug 
0.275 0.35062 0.35062 0.35062 
0.16534 0.10683 0.38183 0.37432 
0.11785 0.12181 0.37216 0.37341 


Example §.3 Solve Laplace’s equation for Fig. 8.5 given below: 


50 = 100 100 =100 ~=650 


Figure 6.5 
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We first compute the quantities u;, w7, wv uw, and w; by using the 
diagonal five-point formula (8.22). Thus, we obtain 


wi? =25.00; uf =42.75; wf) = 43.75; 


wi%=6.25, uf =6.25. 


We now compute vg, uy, v4, and wv, successively by using the standard five- 
point formula (8.20) 


wf = 53.12; wu) 218.75; 


uf? =18.75; uf) = 9,38. 


We have thus obtained the first approximations of all the nine mesh points 
and we can now use one of the iterative formulae given in Section 8.3. We 
give below the first-four iterates obtained by using the Gauss—Seidel formula 
(8.27). 

Lh Ug Uy Uy Lg Ug Uy Ug Uy 


7.06 9.57 708 18.94 2a. 10 18.98 43.02 52.97 42.99 
“13 9.83 7.20 18.81 25.15 16.64 42.94 52.77 42.90 
¥.16 9.88 ¥.18 16.81 2on.08 16.79 42.89 52.72 42.88 
tA? 9.66 7.16 18.78 25.04 18.77 42.88 52.70 42.87 


Example 8.4 Solve the Poisson equation 
iy + Hy, =—10 (x? +37 +10). 


in the domain of Fig. 8.6. 


Figure 8.6 


Let the values of w at the four grid points, A, B, C, D bet w;, mw, u%, 
uy, respectively, Let the grid points be defined by x =ih, y= jh, where A =1, 
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i, j=0, 1, 2, 3. At the point A, 7=1, j=1. The standard five-point formula 
applied at the point A gives 


Wy +o +04+0-—4u, =-10(1+1+10) 
i.€., 


iy = 1 (ay +3 +120). (i) 


Again, the standard five-point formula applied at the point B gives 
uy +g +04+0-4uy =-10(4 +1410) 


Le. 
up == (ty +t, +150) (ii) 


Similarly, the standard five-point formula applied at the points C and D 
gives, respectively: 


i, = (4 +4 +150) (iii) 


Us =< (H +u; +180) (iv) 


From (ii) and (iii), it is seen that yw, =u, and so we need to find only ,, 
uz and uw, from (i), (ii) and (iv). The iteration formulae are therefore given by 


1 
uf) == up? +30 

l n+l n 
uy) =2 [ut ) 4 ul +150} 
Fs (nt) =5 urn ae 


For the first iteration, we assume that wy £0) =u”) =0. Hence we obtain 
u) = 30, 
u ? == (30-+0+150)=45 


AY =; (45) +45 =67.5. 
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quite general but, for easy understanding, we demonstrate its applicability 
with reference to the Laplace equation in two dimensions. For more details, 
the reader is referred to Isaacson and Keller [1966]. 


We consider Laplace's equation in two dimensions, viz., 


Bu Pu 


ae a (8.29) 


and the standard five-point formula 
Meat Tt Mian, p TM pa FM, pot 4M, y =O (8.20) 


The use of formula (8.20) involves the solution of a system of algebraic 
equations, whose coefficient matrix, for n=6, is of the form 


—4 1 0 1 0 0 
l —4 | 0 1 0 
re 0 1 4 0 0 l (8.30) 
] 0 0 4 1 0 
0 1 0 1 —4 l 
0 0 ] 0 1 —4 
The general form of such a system is given by 
rg i 0 
I r I 
B=): : ap (8.31) 
0 I T I 
f fr 
where T is a tridiagonal matrix of the form 
—4 l 
l —4 I 0 
T=| : a ae ; (8.32) 
0 l —4 l 
1 —4 


System A is called a block tridiagonal system and such systems are solved 
by Gaussian elimination or, in the case of large systems, by Gauss—Seidel 
iterations. But tridiagonal system of the type (8.32) are much easier to solve 
than block tridiagonal systems. Hence the question arises as to whether we 
can obtain directly tridiagonal systems in the numerical solution of Laplace's 
equation. Peaceman and Rachford showed that this is possible and their 
method of procedure, called the alfernating direction implicit method (or 
the ADI method) is described below. 
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We rearrange formula (8.20) in either of two ways: 
Ujag, pty, Mig, = By, py, pd (8.33) 
or 
ey pa yg yp = Bp mp, (8.34) 


The ADI is an iteration method and formulae (8.33) and (8.34) are used as 
iteration formulae 


(r+t) (rl) (r4]) _ (r) (r) 
Mi} —4u; at Ss Oe Reel Ot etd BETS (8.35) 
and 
(rl) (rez) (rei) _ (rel) (rel) 
Mi jl ~ 4u i,j OM i pe =e i-i, jf — “s+, J (8.36) 


Formula (8.35) is used to compute function values at all internal mesh 
points along rows and formula (8.36) those along columns. For j =1, 2, 3, ..., 
n=l, Eq. (8.35) yields a tridiagonal system of equations and can easily be 
solved, Similarly, for i=1, 2, 3,...,2—1, Eq. (8.36) also yields a tridiagonal 
system of equations. 

In the ADI method, formulae (8,35) and (8.36) are used alternately, For 
example, for the first row j=1, and formula (8.35) gives 

CE) 4 fo? 40 {GP =u WG}, G=12,3,.00-) B37) 
Together with the boundary conditions, Eqs. (8.37) represent a tridiagonal 
system of equations and are easily solved for hey . We next put j= 2 and 
obtain the values of u 3 ) on the second row. The process is repeated for 
all the rows, viz. up to j=n—1, We next alternate the direction, i.e. we use 
formula (8.36) to compute ui" . It is easy to see that at every stage we 
will be solving a tridiagonal system of equations. Example 8.5 demonstrates 
the method of solution. 


Example 8.5 Solve Laplace's equation, u,, +u,, =0, in the domain of 
Fig. 8.7 (see Example 8.2). 


Yoo yo Uso 
Figure 8.7 
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Substituting the boundary values and solving the above equations, we obtain 


ay _ 8 Qy_ 17 
Wry a ag and ui’ = 75 TOSTTB 


To compute the values on the second column, we now set {/=2 in (8.36) 


Z 2 2 | Lt : 
ef du, +i an == ‘y -u$?, (iv) 
Putting j=1 and j=2 in the above, we obtain the equations 
(2 F 2 1 I 
1% tu ou) aul a 
and 


(2) (2) (2) __,, fl) (1) 
wy Ae HM =U 2G 2 
Substituting the boundary values in the above two equations and solving 
them, we obtain 
uf} =0.1778 and u§} =0.3778 


The iterations are continued to improve the function values obtained first on 
the rows, then on the columns, and so on. The reader is advised to continue 
these computations for the next iteration. 


8.4 PARABOLIC EQUATIONS 


We consider the heat conduction equation: 
C being a constant. (8.38) 


Let the (x, ©) plane be divided into smaller rectangles by means of the sets 
of lines 


x =th, i=, 1, 2,... 


t= jk, j=, I, 2,... 
Using the approximations 


le a Ra (8.39) 
at k 
and 
Ce | 
at = Fy (i, 4 — 2m, y + Mia) (8.39b) 


Eq. (8.38) can be replaced by the finite-difference analogue 


Cc l 
rs (My jap My, = he (my, — 2a + yyy, pds 
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which can be written as 


May ga = My yg + ACM y — 20 p + thy, 7), (8.40a) 
where A = kACh’). 

This formula expresses the unknown function value at the (i, /+ 1)th 
interior point in terms of the known function values and hence it is called 
the explicit formula. It can be shown that this formula is valid only for 
Q<A 1/2. 


For 4 =1/2, Eq. (8.40) reduces to 


l 
Uy j+1 = 3 (uy, jt Mist, fds (8.40b) 


which is called Bender—Schmidt recurrence relation. 

In formula (8.40a), we have used the function values along the jth row 
only in the approximation of O*u/a’. 

Crank and Nicolson proposed a method in 1947 according to which 
a ulax? is replaced by the average of its finite-difference approximations 
on the jth and (j+ 1)th rows. Thus, 


au PLS ie 7 Zu j + ial, j 4 uy 41 t Mist, f4 
ax? 2 i? he 
and hence Eq. (8.38) is replaced by 


Cc l 

Fert i 9 tg FM, gt Mi, god — 2, pe tt, gods 
which gives on rearranging 

~ Atay, jai t (2 + 2A) My jg — AML, jo = AM, T(Z— 2AM, pt Auias, js (8.41) 


where A =kACH"). 

On the left side of (8.41) we have three unknowns and on the right side 
all the three quantities are known, Equation (8.41), which is an implicit 
scheme, is called Crank—Nicolson formula and is convergent for all finite 
values of A. 

If there are .V internal mesh points on each row, then formula (8.41) gives 
N simultaneous equations for the VW unknowns in terms of the given boundary 
values. Similarly, the internal mesh points on all rows can be calculated. 


Example 8.6 Use the Bender-Schmidt recurrence relation to solve the 
equation 
au. Ow 


ig 


ae? = at 
with the conditions 
u(x,0)=4x-x7, su (0, = u(4, 1) =0. 
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Taking h=1, we obtain 
gel Pool. 
2 


Also, w(0,0)=0, wl, 0)=3, w(2,0)=4, u(3,0)=3, and u(4,0)=0. For the 
first time step, k=1. Using the Bender—Schmidt recurrence relation, we 
obtain 


ua =5(0+4)=2, up =5G+3)=3, us = 7(4+0)=2 

For k=2, we have 

my =5(0+3)=15, tn2 = (2+2)=2, Wy =5B+0)=15, 
For k=3, we obtain 

iis =5(0+ 2)=1, my =5(15+1.5)=15 Us 5 =52+0)=1 
With k=4, we have 

ug =5(0+1.5)=0.75, Wp 4 =5(+1)=1, Us 4 = 5(1.5+0)=0.75 

Similarly with £=5, we obtain 
“ =5(0+1.0)=05, up 5 = 5 (0.75 +0.75) = 0.75, iiss =5(1+0)=05, 
The computations can be continued to any number of time steps. 
Example 8.7 Solve 


subject to the initials condition w=sin zx att=0 for 0<x<1 and the 
boundary conditions w=0 at x=0 and x=1 forr>0. Take h=0.2, A=1 
and compute the values of w at the internal mesh points upto two time 
steps. 

We have h=0.2 and A=1, Hence k =h? =0,04, 

The Crank-—Nicolson formula corresponding to 4 =1 is 


it, pd ~ 40, pod + Mie, pod = WE, y Ma, (i) 
Applying (i) at the mesh point #,, we obtain 
O— 4, +. = 0.9511. (ii) 
Again, applying (i) at w., we get 
uy — 4p + uy =-0,5878 — 0.9511 = -1.5389. (iii) 
Similarly, application of (i) at the mesh points uw; and uy gives, respectively: 
‘Wy — 4uy +ug = =0,9511-—0.5878 = -1,5389 (iv) 
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and 
uw, -4u, =-0.9511. (Vv) 


By symmetry, we have uw =n, and wv, = uy. 
Hence, eqs. (ii) to (v) reduce to the two equations: 


4m — uy = 0.9511, My — Say = — 1.5389, 
the solution of which is 
 =0.3993 and wu = 0.6461. 
For the second time step, let ws, ug, uy and uy be the values of w at the 
internal mesh points. Then, applying formula (i) at these mesh points, we 
obtain 
—4us + ug = —0.6461 
— 4u,g + uy = —0,3993 — 0.646] = —1.0454 
Ug — 4uy + tg = —1.0454 
= 4u, =-0.6461. 


By symmetry, — lig se lig =uz. Hence the above equations reduce to the 
two equations 


—4us +ug =-0.6461 and us —3ug, =—1.0454, 
the solution of which is 
us =0.2712 and wu, =0.4387. 


Example 8.8 Solve the heat equation: 


subject to the conditions 
u(x,0)=0, u(0,)=0, u(LA=r 


(i) We first choose k= 1/8 and 4=1/2 so that A=kih* =1/2. The Crank- 
Nicolson scheme (8.41) now becomes 


~My, je F Gay 41 — M41, jel = 4-1, 5 * ot; y+ Mya, y (i) 


Let the value of w corresponding to ¢=1/8 and x=1/2, i.e. at the mesh 
point P be wu, (see Fig. 8.8). Applying the Crank—Nicolson scheme (i) given 
above at this point, we obtain 


0+6y, - j =0 which gives uw, = 0.02083. 
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Figure 6.6 


(ii) We now choose k=1/8,h=1/4 so that A4=2 (see Fig. 8.9). The 
Crank—Nicolson scheme corresponding to this value of A is given by 


Uy, pad + 3, fe — Mit, pd = Mig Mi, yg + a, yp (ii) 


Applying the above equation at the mesh point P, we obtain 


O+3u,-—w,=0 or 3, =u, 


x % 
— uv=0 
Figure 6.9 


Similarly, applying the same equation at the mesh points Q and R, we obtain 
the two equations 


=u, +3u,-u,=0 and -w,+3u -i=0, 
We have thus three equations in the three unknowns w,, wu, u, and the 
solution is 


uy =0.00595, uv, =0.01785, —u; = 0.04760 


(ili) As our final choice, we choose £=1/16, h=1/4 so that 4=1. This 
means that we propose to find our solution for f=1/8 in two steps instead 
of one as in (i) and (ii) above. 
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The Crank—Nicolson scheme corresponding to this value of A is now 
Ua jad + 4, pa — Mist jon = Yi-1y + May (iii) 
Applying the scheme (iii) above at the mesh points P, Q and R, we obtain 
the three equations: 
4u—u,=0, wu, +4u, -—u, =0, my + 4s - 2 =0 
whose solution is 
ii, = aE iy = 5 
1 56x16)? S6x4" 3 56x16 
Again, applying the scheme (iii) at each of the mesh points X, Y, Z in 
Fig. 8.10, we obtain the three equations: 


1 
1M 6 


_ +d4y, = ik 
Mg ts — ig 56- 


fA, 
4x56 16 


a le ig 


Figure 8.10 


The solution is 
4 =0.005899, us =0.019132, ug = 0.052771 


The exact solution of the problem is given by Froberg: Introduction to 
Numerical Analysis, p. 269. 


i 1 
u(x, =o? —x+6xf)+ Es > Cy" ene sin ruarx 
n=l 


rv 


*. 
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which gives 


l | 1 1 3 1 
41, 0.01878, 0 
“(5 i)- 0.005 u(3. i)- u(2. i= 05240. 


8.5 ITERATIVE METHODS FOR THE SOLUTION OF EQUATIONS 


The iterative methods discussed in Section 8.3 can be applied to solve the 
finite-difference equations obtained in the preceding section. In the Crank— 
Nicolson method, the partial differential equation 


uu 
at ax? 
is replaced by the finite-difference equation 


I 
(Lr) iy p44 = My, y + ori, p01 + Uist ft Mie, i tM y 2m, 4) (8.42) 


where r = K/h’, 

In Eq. (8.42), the unknowns are uj 54), 4%), ja ANd wj4y j.1, and all 
others are known since they were already computed at the jth step. Hence, 
dropping the /’s and setting 


] 
cy = uy ij +r (in, j = 2uy yp +My, y) (8.43) 
Eq. (8.42) can be written as 


Fr Cc; 
cnvee). eee an (8.44) 
_ 2(1+r) a+ Min) + l+r 


From Eq. (8.44), we obtain the iteration formula 


: 

u (ne) oa (n+ we (8.45) 
which expresses the (n+1)th iterate in terms of the mth iterates only, and 
is known as Jacobi's iteration formula. 

It can be seen from Eq. (8.45) that at the time of computing u | pat 
the latest value of u,_,, namely u, ley is already available. Hence, the 
convergence of Jacobi’s iteration cml can be neatly by replacing 2 “7 
in formula (8.45) by its latest value available, namely by as ) Accordingly, 
we obtain the formula 


l | rt 
Megane tHMtT en 
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which is called the Gawss—Seidel iteration formula, It can be shown that the 
scheme (8.46) converges for all finite values of r and that it converges 
twice as fast as Jacobi’s scheme. 

Equation (8.46) can be rewritten as 


1 F i n c n 
uf aff Ell? ou tds ul} 
from which it is clear that the expression within the curly brackets is the 
difference between the mth and (m+ 1)th iterates. If we take the difference 
to be @ times this expression, we then obtain 


1 Fr i c 
uf? auf v0n| (ult afte Seu | (8.47) 


which is called the suecessive over-relaxation (or SOR) method. w is called 
the relaxation factor and it lies, generally, between | and 2. 


Example 8&9 Solve 


subject to the initial condition u=sin zx at r=0 for 0s x<1 and w=0 at 
x=0 and x=1 for ¢>0, by the Gauss—Seidel method. 

We choose h=0.2 and &=0,02 so that r=A/h* =1/2, The formula 
(8.46) therefore becomes 


uf = eu? +u Nh S 4 @ 


Let the values of wu at the interior mesh points on the row corresponding to 
t=0.02 be w,w>,u3, ug, as shown in Fig. 8.11. 


t 


u=0 u=0 


wool PP 
4 0.6 0.8 1.0 


0 02 
u 
—~— 60 OSB76 09511 058768 09511 0.0 
Figure 8.11 


x 
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Applying the formula (i) at the four interior mesh points, we obtain successively 


4) , [0+ $”]+ : jo.sers + =(0- 2x 0.5878 +0.951 »| 


= Ly”) 403544 (ii 


yr =< (wel) +u$}+2/ 0951 +2 (05878 ~20.9511+40.951 1) 


= [uf + $]+0.5736 (ii) 
(eel) I (n+l) (re) 2 l 

us) = elu? +g ]+ 5 | 0.95114 (0.9511 -2 0.9511 + 0.5878) 

pu SY + 0 8] +0.5736 (iv) 


7 (n+l) = : [u fr) +O0]+ : ose7e + ~(0951 1-2 0.5878 + 00) 


: us") 40,3544 (v) 
Formulae (ii), (iii), (iv) and (v) can now be used to obtain better approximations 
for wu), W2, v3; and wy, respectively. The table below gives the successive 
iterates of uw), uw, uw, and wy corresponding to f=0.02. 


x 0.0 0.2 0.4 0.6 08 1.0 
ux) 0.0 0.5876 0.9511 0.9511 0.5878 0.0 
n=0 0.0 0.5876 0.9511 0.9511 0.5878 0.0 
n=1 0.0 0.5129 0.8176 0.8078 0.4890 0,0 
nee 0.0 0.4907 0.7900 0.7868 0.4855 0.0 
n=3 0.0 0.4861 0.7858 0.7855 0,4853 0.0 
n=4 0.0 0.4854 0.7854 0.7854 0.4853 0.0 
n=5 0.0 0.4853 0.7854 0.7854 0.4853 0,0 


The symmetry of the solution about x =0.5 is quite clear in the above table. 
The analytical solution of the problem is given by u=e™* ' sin rx and the 
exact values of w for x=0.2 and x=0.4 are respectively 0.4825 and 0.7807. 
The percentage error in both the solutions is about 0.6%, and the error can 
be reduced by taking a finer mesh. The reader should check some of the figures 
given in the table.* 


*For the derivation of a more general finite-difference representation of the 
parabolic equation, see the paper by Sastry [1976]. 
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8.6 HYPERBOLIC EQUATIONS 


We consider the boundary-value problem defined by 


ty = C7 tt, (8.48) 
u(x, 0)= f(x) (8.49) 
u, (x, 0) = 4(x) (8.50) 
u(0,.9=y,(t) (8.51) 
u(1, t) = y(t) (8.52) 


for O<sr¢<¢T, which models the transverse vibrations of a stretched string. 
As in the previous cases, we use the following difference approximations 
for the derivatives 


l 
i, = TAG, —2u, ; + j1, ;) + O(h*) (8.53) 
and 
l 
a Aa (14, j-1 — 24, +H, jx) FOC), (8.54) 


where x =ih, i=0, 1, 2,..., and t= jk, j=0, 1, 2,... 
Further, u,(x,f) is approximated as follows 


u, (x, p= tt +00) (8.55) 


Substituting (8.53) and (8.54) in (8.48), we obtain 


Z 
ath, jam 2t, y+, j= =a 7 tM, 


Putting a =ck/h in the above and rearranging the terms, we obtain 


My ja] = My yy tO (Mpg +My y+ 2-27) (8.56) 


Formula (8.56) shows that the function values at the jth and (/— 1)th time 
levels are required in order to determine those at the (f+ 1)th time level. 
Such difference schemes are called three level difference schemes compared 
to the two level schemes derived in the parabolic case. 

By expanding the terms in (8.56) as Taylor’s series and simplifying, it 
can be shown that the truncation error in (8.56) is O( + A’). Further, formula 
(8.56) holds good if a <1, which ts the condition for stability. 

There exist implicit finite difference schemes for the equation given by 
(8.48). Two such schemes are 
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My j+1 au, j Uj 7-1 Po 
er ca = a Ps i ae ee 


a oo a (8.57) 
and 


My jal — uy; py ce 


Pz = 4he (CA wes) a uj f+] + Wy j+1) 
+ 2(Uj yyy 2m, jp +My) 
+ (upg pn 20 py + Mp (8.58) 


Formulae (8.57) and (8.58) hold good for all values of ck/h, The use of 
formula (8.56) is demonstrated in the following examples. 


Example 8.10 Solve the equation 


atu a 
oe at 


subject to the following conditions 
u(0,=0, w(lt=0 ¢>0 
and 


2H (x,0)=0, u(x, 0) =sin? (2x) for all x in OSxS1. 
This problem admits an exact solution which is given by 
u(x,f)= : sin #x COS At — ; sin 32x cos Jat (i) 
We use the explicit formula given by (8.56), viz., 
Uy pet =— My, jy tO (Uy, p HU, )+2C-@*)u, , where a= <1 Gi) 
Let h=0.25 and k=0.2. Hence a =0.2/0.25=0.8, so that the stability 


tennehaitg is satisfied. Let u, =w(ih, jk), so that the boundary conditions 
ecome 


Uo, =0 (iii) 
My p= 0 (iv) 
u,g =sin*(xih), i=l, 2,3, 4 (v) 
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and 
My 1 =0 so that a _) =u). (vi) 
Substituting the value of a =0.8, eq. (ii) becomes 
Uy jay = Uy, jy + 0.64 (uy_y y+ yyy, ») + 2(0.36) wy, , (vii) 


At the first step, /=0 and the above equation becomes 
Ui; | =k -1 + 0.64 (uj; + jy) )+2(0.36)u, , 


Or, 
My y = 0.32 (uy, 7 + ty, )) +036; |, (viii) 

using (vi) 
Hence my) = 0.32 (up » + H2,9)+ 0.361 9 

= 0.32 (0 + 1) + 0.36 (0.3537) 

= 0.4473. 
The exact value w(0.25, 0.2) = 0.4838. 

Again, 


Uy, = 0.32 (0.3537 + 0.3537) + 0.36 (1.0) = 0.5867 

Exact value = 0.5296. 

Finally, 

uy; = 0.32 (1.0 + 0) + 0.36 (0.3537) = 0.4473 

Exact value = 0.4838. 

The computations can be continued for j=1, 2,... 
Example &11 Solve the boundary-value problem u, = 4u,, subject to the 
conditions: 

u(0,)=0=n(4, 1), u, (x, 0) =0, u(x, 0)=4x—x?. 
We take h=1 and a=1 so that £=1/2=0.5. 
Since u(0,f)=u(4,1)=0, uw vanishes from x=0 to x=4, i.e. 
upp =M4,=0 — for all j. 

Again, since w,(x,0)=0, we have 


a i 
vie + 


uy | —~ ty =) for jz. 
The above relation shows that the values of uw are the same for j =1 and 
jz-l. 
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Finally, u(x, 0)=4x- x? gives 
u, 9 =4i-i", since h=1. 
Then 
M,0=9 Mo=3, Mo=4, 039 =3, 
Now, for a@=1, Eq. (8.56) becomes 
My ja = My, ja tM, 7 + M41, 
For j=0, the above relation gives 


My y = yy Fj oF Bi, 0 
or 


361 


Mao =0. 


(i) 


l a 
4,1 => (Mio + Meo — Since uy) =, .1. 


From Fig. 8.12, we obtain 


l l 

M1 => (Moo tM2,9)=5 OF 4)=2, 
1 

af A = Gta 
| 

Wy, = 5 (440) =2. 


For k=l, we use eq. (i) with j=1: 


yy = ty g + jy | + May 
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CHAPTER 


Numerical Solution of 
Integral Equations 


9.1 INTRODUCTION 


Any equation in which the unknown function appears under the integral sign 
is known as an integral equation. When the limits of the integrals are 
constants, we have a Fredholm equation. For example, equations of the form 


b 
| K(x,t) f(t) dt =@(x) (9.1) 


b 
A [ Keo f(t) dt = f(x)+ @(x) (9.2) 


are called linear Fredholm integral equations of the first and second kina's 
respectively. In each case the unknown function is f(x) and occurs to the 
first degree, @(x) is a known function and the kernel! A(x, f) is also known. 
If the constant 6 in Eqs. (9.1) and (9.2) is replaced by x, the variable of 
integration, the equations are called Volterra integral equations. For example, 


A [ Keo F(t) dt = f(x)+ Px) (9.3) 
is the Volterra integral equation of the second kind. 


If @(x)=0 in Eg. (9.2), then the equation is called homogeneous, otherwise 
nonhomogeneous. For nonhomogeneous equations, A is a numerical parameter 
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whereas for homogeneous equations it is an eigenvalue parameter because 
in such a case the integral equation presents an eigenvalue problem in which 
the objective is to determine those values of A, called the eigenvalues for 
which the integral equation possesses nontrivial solutions called eigenfunctions. 
If the kernel A(x, f) is bounded and continuous, then the integral equation 
is said to be nonsingular. If the range of integration is infinite, or if the 
kernel violates the above conditions, then the equation is said to be singular. 
To solve an integral equation of any type is to find the unknown function 
satisfying that equation. In practical cases, however, the solution of an 
integral equation by analytical techniques is out of the question, and hence 
it would be necessary to adopt a numerical method of solution. 
Fredholm integral equations, particularly those of the second kind, occur 
quite frequently in practice and hence, we restrict ourselves, in this chapter, 
to a brief treatment of numerical methods for the solution of nonhomogeneous 
linear Fredholm integral equations of the second kind. Before presenting 
these methods, it would be instructive to demonstrate the relationship between 
integral equations and initial-value problems. This is shown in Example 9.1 


Example 9.1) We consider the initial-value problem 


y"+y=0, with »(0)=0 and »'(0)=1 (9.4) 
Let 
rl 
a= H(0 (9.5) 


Integrating both sides of (9.5) with respect to x, we obtain 
<. = i} u(x)dx + y'(0) = j u(x) dx + 1, 
0 0 
on using the given condition. Integrating the above with respect to x, we get 
x 
y(x) = [ @-pudrsx (9.6) 
0 
Substituting (9.5) and (9.6) in (9.4), we obtain 


u(x) + [ @-puydr+x=0 
i] 


or u(x) =—x + j (t ~ x) ult) de, 
0 


which is a Volterra integral equation. 
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9.2 NUMERICAL METHODS FOR FREDHOLM EQUATIONS 


There exist several methods for the numerical solution of Fredholm integral 
equations of the second kind, e.g. method of degenerate kernels, method of 
successive approximations, collocation and product-integration methods, etc. 
We present a few of these methods, in a formal way, with simple examples. 
For error estimates and other details, the reader is referred to Atkinson 
[1971]. 


9.2.1 Method of Degenerate Kernels 
We consider the integral equation 


f{x)- | K(x, f(t) dt =x (9.7) 
A kernel A(x, £) is said to be Jeaeiat if it can be expressed in the form 
K(x, = 3 u(x) v, (f) (9.8) 
Substituting this in (9.7), We ee 
fo)-¥) [uc v(t) f(t) at = G(x) (9.9) 
Setting ; 
b 
Ju @s@d=4 (9.10) 
Eq. (9.9) gives ; 
foy= A, u;(x) + A(x) (9.11) 


The constants A, are still to be detemined, but substituting from (9.11) in 


(9.10), we get 
b 
| v(t | 
a J 


it iad 


Aju (ft) + p(t) 
| 
Or 
fn 5 b 
> A, [v(t u(t) dt+ [1 o@ de= 4, (9.12) 


which represents a system of m equations in the nm unknowns A), A),..., 
A, When the 4; are determined, Eq. (9.11) then gives f(x). 
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Although the method is important in the theory of integral equations, it 
does not seem to be much useful in the numerical work, since the kernel 
is unlikely to have the simple form (9.48) in practical problems. In general, 
however, it is possible to take a partial sum of Taylor's series for the kernel. 
This is shown in Example 9.3. 


Example 9.2 We consider the equation 
wid 
f@)-a j sinx cos t f(t) dt =sinx. 
0 


Setting 
mia 


j cos tf() dt = A, (i) 
0 


the integral equation becomes 
f(x)=AAsinx+sinx =(AA+1) sin x. 
Substituting this in (i), we obtain 
ail 
i) cost(A.4+1)sin¢ dit = A, 
0 
which gives on simplification 
fell 
=o 
Hence the solution of the integral equation is given by 
f(x)= — ssing (242) 
By direct substitution the reader should verify that this is the solution of the 
given integral equation. 
Example 9.3 Solve the integral equation 


I 
fa)=5e* +3x-1)+ | (eo —1)x f(t) de. 
0 


We have 
K(x,)=(e" —nx 


4 
-{1-x? Ey} 


=P lt, 
neglecting the other terms of the Taylor’s series. 
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Hence the given integral equation becomes 


| 
f()= ste +3x-1+ (2? + tet) f(at 
0 


I | 
=5(e* +3x-1)- 7 JP sod ris “sear 


aii +3x-I)- kx? + th, 
2 2 
where 


I 
Kye [?foae 
0 


1 
K,= ey f(Ode. 
0 


Substituting for /(f) from (i) in (ii), we obtain 


l 
K, = ie 3c" +3'-l)- nr +t Kat |e 
0 


Since 
5 


1 
| edt =2--, 
0 é 
eq. (iv) gives 

6K,_K,__ 5 ,29 


§ 12 2 24 


Similarly, substituting for f(r) in (iii) and simplifying, we obtain 


Solution of (v) and (vi) is given by 
K, =0.2522 and K, =0.1685. 


Hence the solution of the given integral equation is 


f(x)= ; (e7* +3x-1)-0.2522x* + 50. 1685)x°, 
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(i) 


(ii) 


(iii) 


(iv) 


(v) 


(vi) 
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§.2.2 Quadrature Methods 


We consider the integral equation in the form 
b 
f(x) [ KG,9 SO dt = 60. (9.13) 
a 


Since a definite integral can be closely approximated by a quadrature formula, 
we approximate the integral term in (9.13) by a formula of the form 


J rea x A, F (Xm) (9.14) 


m= | 


where A,, and x,, are the weights and abscissae, respectively. Consequently, 
(9.13) can be written as 


S(x)- YAK ty )F () = OC), (9.15) 


m=! 


where #),f,...,f, afé points in which the interval (a, 5) is subdivided. 
Further, Eq. (9.15) must hold for all values of x in the interval (a, 5); in 
particular, it must hold for x=4,,x=%,...,x=f,. Hence we obtain 


LOQ)— YAK Gt) Ln = OG)» 1=12,..2 (9.16) 
m=] 


which is a system of m linear equations in the » unknowns /(t,), /(f), 
woo S (t,). When the f(t,) are determined, Eq. (9.15) gives an approximation 
for f(x). Obviously, different types of quadrature formulae can be employed, 
and the following examples demonstrate the use of trapezoidal and Simpson's 
rules. 


Example 9.4 Solve 


j 
3.39 
f(x)- Jernsoar=>x ; (i) 


By direct substitution, it can be verified that the analytical solution is 
given by /(x)=x-—I1. For the numerical solution, we divide the range [0, 1] 


into two equal subintervals so that h = 1/2. Applying the trapezoidal rule to 
approximate the integral term in (1), we obtain 


fia)-4] «+2 +4 Jirorng|-ds -2, where f; =f (x;). 
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Setting x=1,, where fp =0, f =1/2 andr, =1, this gives the system of equations 
12 fo - 34, 3/2 =—10 
-3 fp +124, -9f =-2 
“Ifo -9)f, +6), =8 


The solution is 


l 5 1 
jo= > A= 6? fi=5- 
On the other hand, if we use Simpson's rule to approximate the integral 
term in (i), we obtain 


5 it 
fa)-2 xh +4( +3) rorop|a3x-3 (il) 


Setting x=r,, we get 
6fo -2f-f2=-5 
-fo9 +4f,-3f) =-1 
=f) -6/, +444 =4. 
The solution of which is 


fo =-l, fi=-5. fo =0. 


Using these values in (ii), we get 


=x—1l, which is the exact solution. 


It should be noted that Simpson’s rule gives exact result in this case since 
the integrand is a second-degree polynomial in ¢ 


Example 9.5 The integral equation 


1 
y(x)+ | Xe, s) y(s) ds =1, (i) 
=I 
where 
| ] _ 
K(x, s)=— ———- (ii) 
ad ® 1+(x=sy 
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occurs in an electrostatics problem considered by Love [1949], and is called 
Love's equation. The analytical method of solution, suggested by Love, is 
somewhat laborious and various numerical methods were proposed. The 
simplest is to approximate the inegral term in (i) by the trapezoidal rule. For 
this we divide the interval (-1, 1) into m smaller intervals of width A, the 
ith point of subdivision being denoted by s,, such that 


s; =-1+ ih, i=Q, 1, 2,....7 
and mh=2. Denoting y(x,;) by y,, eq. (i) gives 
a-1 4/+! 
w+ Df Kes) y(s)ds=1. 
j=0 aj 


Approximating the integral term by the trapezoidal rule, the above equation 
becomes 


A-| 
H+ YD, SL KG5/HG))+ Kis pp) ] = 
j=0 


which can be rewritten as: 


h h sis 
¥; +5 KGi50)¥% +5 KG Sn)In th > K(x,,5,)y; =1 (iii) 
l 


for i=0, 1, 2,....m. Equation (iii) represents a system of (m+ 1) linear 
equations in (#+ 1) unknowns, ViZ., Yo, }y.-.-.¥,_. and was solved on a 
digital computer. The solution is symmetric and the computed values of 
y(x) at x=0 and x=lare given in the table below. For comparison, the 
exact values are also tabulated. To study the order of convergence of the 
method, computations were made with different values of n. The A’-order 
of convergence of the trapezoidal rule is quite revealing. 


x Exact yx) n Computed yx) Error Ftatio 
0.0 0.65741 4 0.66026 0.00285 
8 0.65812 0.00071 4 
16 0.65759 0.00018 4 
a2 0.65746 0.00005: 3.6 
1.0 0.75572 4 0.75452 0.00120 
8 0.75542 0.00030 4 
16 0.75564 0.00008 3.75 
2 0.75570 0.00002 4 


9.2.3 Use of Chebyshev Series 
We consider the Fredholm integral equation in the form 
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9.2.4 The Cubic Spline Method 
We know that in the interval x), =x5x,,3(x) is given by 


(xy-xP 0 G44) 
—————— + Af pe Se es 
6h J 6h 


im x,-x he X= X54 
fra 1] ; o{,- a] oe eee 


where M, =s"(x,), ¥; = y(x,), and x, =Xxq + jh, j =0,1,....N. If we now 
approximate the integral term in (9.17) by using (9.33), we obtain 


s(x) = 


_ee = 3 
Kar+D, { Ko] My (s;—s) +M = Sjay 


- 6h 4 6h 
= sy 
h* s,78 h* (s—5;-) 
— -—M ds 
{oy 6 | h ly, 6 , Fr 
= f(x), €=0,1,2,...,M (9.34) 


Putting s = 5,4 + ph, the above equation simplifies to 


yo 2 3,2 
(= pyh* poh 
meh, J Kees + ph) a = BPI ‘el og 


2 
{a eu, Je “prof y,-Em, Jole 


=f(x,), i=0,1,2,...,N (9.35) 
In (9.35), the integrals 
1 
[ K (x51 + ph) p™ dp,  m=0, 1,2 and 3, (9.36) 
0 
have to be evaluated. This can be done either analytically (wherever possible) 


or alternatively, by numerical techniques. When these integrals are evaluated, 
Eqs. (9.35) together with the relations 


h th h _ yp 7 2¥y + Yip 
=M —M,+—M cf eo Ee a 
6.0 gg h 
j=l, 2,...,N-1 (9.37) 
and Me =My = 
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will form a set of (2 + 2) linear algebraic equations in (2N + 2) unknowns, 
WiZ., Vo Poe Fs Mg,.Mj,....4¢y. As an example, we consider again 
Love's equation given in the previous example. 


Example 9.7 In contrast with the previous methods, the spline method 
can be applied when the values of d are small. For this particular example, 
the integrals in (9.36) were calculated analytically. Thus for m = 0, we have 


1 1 
] d 
Xq = [ KGxp 51+ ph) dp=+ ee fe 
0 : Jerre ja 7 Phy 
Putting x; =—-1+ih and S54 ==] +(/—1)4, and evaluating the definite integral, 


we obtain 


Ag ~Lwe'| 2 ft ss a | 
aT 1+(A° la" i= | Gi- f+) 
Similarly we obtain the results 


1 
X= | KG, 5/1 + php dp 
0 


d d* 4h G- jy 
se ge he 
2ch* d? +h - j +1) ” 


1 
Xy= [ K(%,5,1+ ph)p* dp 
O 


Z 
SS po0? fx + 2X, (i- 7 +1) 


| 
Xy= | Ktsinsya + ppd 
0 


2 
= otseMi~ ple] aij? lx 


2 

=32741) Is 16-740" |X 
The system of equations was solved by the Gauss-Seidel iteration method 
and a standard subroutine was used for this. The results are summarized in 
the following table for different values of ¢d, and agree closely well with 
those obtained by Phillips [1972]. It was found that the method is unsuitable 
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for finding the solution for larger values of d as the convergence is rather 
slow. Thus for d= 1.0 the value obtained with 500 iterations for x = 1.0 is 
0.80692 compared to the true value 0.75572. For more computational results, 


see the paper by Sastry [1975]. 
Cubic Spline Solutions of Love's Equation 


yx) 

x d=0.1 d=0.01 d= 0.001 * 
0.0 0.51261 0.50146 0.50015 
0.2 0.51470 0.50158 0.50016 
0.4 0.51858 0.50187 o.50019 
0.6 0.52876 0.50261 0.50026 
0.8 0.60688 0.51713 0.50271 
1.0 0.78627 0.69641 0.67179 


These results show that the spline method for the numerical solution of 
Fredholm integral equations is potentially useful. Its application to more 
complicated problems will have to be examined together with an estimation 
to error in the method, It seems probable that the condition of continuity 
of the kernel may be relaxed, and the advantage to be achieved by using 
unequal intervals may also be explored. Finally, the solution obtained by the 
spline method can be improved upon by regarding it as the initial iterate in 
an iterative method of higher order convergence. 


9.3 SINGULAR KERNELS 


If K(s,¢) is discontinuous or continuous but badly behaved, the integral 
equation is called a singular integral equaion and the quadrature methods, 
discussed earlier, should not be applied. We may, however, approximate the 
smooth part of the integrand by a simple function and then integrate the 
total new integrand exactly. Such formulae are called generalized quadrature 
formulae, also called product integration formulae. 

We consider the integral equation 


& 
fe+ [KOO fOdt=9@, asxse, (9.38) 
ad 


Let b-a=nh and t,=a+ jh, /=0,1,...,.4 80 that f9 =a and ¢, = 5, Then 
(9.38) can be written as 


a-1 {y+ 


fa+y f KO sOd =H, (9.39) 


=O) 
i= oi 
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I =| h 
= — lan 
he ease 


and 
I 
X1G A= J PKG .t; + Ph) dp 
0 
1 
=1 f — 28, 
m 1+h(i-j-py 
1+A°(G— j-17 ae 
= —— og) 7 i pxon, 
2h? x Dea Mineraeete 
then 


ay =h[XOG,/)-AIG,/)) and By = AX 1G, /). 
With n= 4, we obtain from (9.41) the equations 
1.076 fy + 0.126 f, + 0.081, + 0.050 f, +0.018/, =1.0 
0.071 fo +1.153f; +0.126f5 + 0.0815 + 0.029 f, =1.0 
0.047 fo + 0.126 f, +1.153f4 +0.126 f; + 0.047 f, =1.0 
0,029 fo + 0.081 /, + 0.12675 + 0.1534 + 0.071 /4 =1.0 
0.018 fo +:0.050 7, + 0.08175 + 0.1267, +1.076/4 =1.0 


The solution of this system, which is centro-symmetric, was obtained on 
a digital computer. The computations were repeated for n= 8, 16 and 32 
and the results, together with the exact values, are tabulated below: 


x Exact yx) n Computed A x} Error 

0.0 0.65741 4 0.65609 0.00132 
8 0.65708 0.00033 

16 0.65733 0.00008 

a 0.65739 0.00002 

1.0 0.75572 4 0.75484 0.00088 
8 0.76550 0.00022 

16 0.75566 0.00006 

a2 0.75570 O.00002 


Comparison with the results obtained by the ordinary trapezoidal rule 
(see table of results in Example 9.5) shows that this rule gives better 
accuracy than the ordinary trapezoidal rule. The order of convergence is A’ 
as in the latter rule. 
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The next example demonstrates the use of generalized quadrature in 
dealing with kernels having a logarithmic singularity. 
Example 9.9 We consider now an example from fluid mechanics involving 
potential flow of an incompressible inviscid fluid. 

In many fluid dynamics problems, it is nécessary to calculate the pressure 
distribution on the surface of a body moving in a fluid. For a body of 
revolution in axial flow, Vandrey [1961] derived the linear integral equation 


£ 
v(s) = 2x'(s) -+ K(s,o)v(o)do, O<s<L (i) 
0 
where 
K(s, 0) = ey K(k) 
J -8? +(v+ny } 


~ E(k)| 2-4) , up ted | (ii) 
y Te BP +n)? 


4yn va 
(x-éy +(y+ny as 


and A(k) and £(«) are complete elliptic integrals of the first and second 
kinds respectively with modulus &. In (i), v(s) denotes the velocity distribution 
function on the body surface from which the pressure distribution can be 
found by Bernoulli's equation. Details of the problem and its reduction to 
a system of equations are given in the papers by Kershaw [1971] and Sastry 
(1973, 1976], where further references may be found. Using the expansions 
of K(k) and E(é) given in Dwight [1934], the kernel K(s,o) in (ii) can be 
split into the form: 


K(s, 0) = P(s, 0) log|s -o|+Q(s, 7) (iii) 


K, = 


where 


l xy-y(x-§) 2 
(sre 2 py, 


P(s,0)=- 5 ; 
Ve-8 +(y+7) ¥ 


x'(y-n)—y'(x-€) 2 (iv) 
rea rae —[K(k,) — £0, )] 
(x-€ +(y-7 


(3,0) = A(s,o)— Pls, oo) log| s, | 
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and 


_ ey _: 
ap-P -EoeT On 
(x-E)* +(y +7) 
When o=s, it is found that 


x 
P(s, 8)= ay 
a (v) 
Xs, ont e[-Lioes y?)+plogdy*+logt—i| y= 2-2 Mit 
2y 2 2 xi? 4 y”? 
The method of generalized quadrature described in Chapter 5 can now be 
applied to reduce the integral equaion (i) to a system of linear algebraic 
equations. 

The table below gives the numerical results for a cylinder. As the 
solution is centro-symmetric, the results are given only up to s=90°, The 
computations are made with 20 subdivisions and the accuracy is quite good. 
For the sake of camparison, the accurate value 1.5 sins is also tabulated. 
On running the program twice with n= 10 and m= 20, it was found that 
the order of convergence is two. 


Accurate value 
& (in deg) of vs) Computed value Error 
18 0.4635 0.4619 0.0016 
36 0.8817 0.6816 0.0001 
4 1.2135 12141 0.0006 
72 1.4266 1.4275 0.0008 
90 1.5000 1.6011 0.0011 


For a numerical solution of this problem using Everett's formula, see 
Kershaw [1961]. 


9.4 METHOD OF INVARIANT IMBEDDING 


This is a method of recent origin, being mainly due to the efforts of Kalaba 
and Ruspini [1969], and is applicable to Fredholm integral equations of the 
second kind 


v(x) = g(x)+ | K(x, 8) Ws) ds (9.43a) 
vi] 
where 
K(x,s)= | f(x2) f (sz) w(z) dz (9.43b) 
0 
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In the method of invariant imbedding, Eq. (9.43) is first rewritten as a 
Volterra integral equation in the form 


y(x, 1) = g(x)+ [xe» y(s, t) ds; Osxst, O<rea. (9.44) 
i] 


An essential feature of the method is to convert the Volterra integral equation 
(9.44) into initial-value problems and then solve the initial-value problems by 
any of the standard techniques. The transformation to the initial-value problems 
involves a series of complicated mathematical manipulations and the interested 
reader is referred to the original paper by Kalaba and Ruspini [1969]. We, 
however, demonstrate its applicability to a practical situation. 


Example 9.10 We consider the problem proposed by Srivastava and Palaiya 
[1969] who have studied the distribution of thermal stresses in a semi- 
infinite solid containing a pennyshaped crack situated parallel to the free 
boundary. The free boundary of the solid is kept at zero temperature and 
in the axisymmetric case the problem is reduced to the solution of a Fredholm 
integral equation of the second kind 


if 
wx) + | K(x, 5) y(s)ds = 5, () 
a it 
where 
ea J e254 cos Ex cos Es dé, (ii) 
aT 
o 


in which y(x) represents the non-dimensionalized stress distribution function 
and the integral equation was derived by assuming that the centre of the 
crack is at the origin; that the solid, which is isotropic and homogeneous, 
is divided into two domains: (i) the layer defined by -—H 4. z<0, and (ii) the 
half-plane 05 zS 9; and that the temperature prescribed on the surface of 
the crack is constant. The derivation and physical details of the problem 
may be found in the above cited reference where the integral equation was 
solved by the classical iterative method for small values of the ratio of the 
radius of the crack to that of its distance from the free boundary, and for 
values of this ratio mearer unity, the equation was solved numerically by 
quadrature method. 

For the numerical solution by the method of invariant imbedding, the 
radius of the crack is assumed to be of unit length and the integrals are 
approximated by using Gaussian quadrature. Then, the initial-value problems 
become: 
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da) omen 3 to, oe Fa dy) Ra) 
- r (iit) 
ay een Faden Ra 
Ry(0)=0 
and 
dato -|s0-¥ 2 5 Fad cone) 
(iv) 


m=] 


, 2 
cor = shepnineinal 
e,(0) = 0, isisN, OSrel. 


where 


e(t)=e(4;,f) 
and finally, 


y(x, 1) = a+ Ta AM An) 05% en 0<x<rs1.(v) 
m=] 


In (ili) to (v), the notation 
aoe 
= l+a, 


is used, a,, and F,, being the abscissae and weights of the N-point Gaussian 
quadrature formula defined by 


I N 
J f@ a=) FS Gn) 
-| m=] 


The eqs. (iii) and (iv) have been solved using the fourth-order Runge-Kutta 
method, and the five-point Gaussian formula. The results are obtained on 
a digital computer and are given in the following table for different values 
of A: 
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H x ya) 
1.06 0.0 1.7718 
1.0 =1.7013 
1.1 0.0 —1.7450 
1.0 —1.6813 
1.2 0.0 -1.6898 
1.0 —1.6464 
1.3 0.0 ~1,6599 
1.0 —1.6169 
1.6667 0.0 -1,5618 
1.0 -1,5397 


Although the method produces results which agree quite well with those 
obtained by Srivastava and Palaiya, it suffers with the serious disadvantage 
of being a complicated process and requiring an enormous amount of computing 
time. 

A central idea of the method is to take full advantage of the ability of 
the modern highspeed digital computer to solve systems of ordinary differential 
equations with given initial conditions, and it therefore finds important 
applications in the numerical solution of integral equations occurring in 
radiative transfer, optimal filtering and multiple scattering. 


EXERCISES 


9,1. Verify whether the functions given below are solutions of the integral 


equations indicated against them: 


l 
(a) fe) =1: f(x)+ | xe -1) SO dt=e* -x 
0 


1 
(b) w(t) =e u(t)+A | sin(¢x) u(x) de = 1 
0 


? i 
7 xX it _* 
(c) F@)=sin>:f@)-> J K(x,1) f() dt == 


s(1-), Osxst 
Z 


{1-2}, texel 
2 


where 
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| 
(d) @(x)=x: W(x)= =" 4 [+o¢mar 
oh 
i 3 5 
(e) freA TO | GNI Res 5, 


9.2. Solve the following integral equations with degenerate kernels: 
ald 
(a) f@)-a [tans f(s) ds=cotx, 
—m/4 
ail 
(b) f(x)-a sinx cost f(t) dt =sinx. 
0 


(c) f)-af sin(x —u) f(u) du =cos x. 
0 


1 
(d) f(x)=sinx+ | (1-xcos(xt)) f(t) dt 
0 


9.3, Solve the integral equations given in problem 1(d) and (e) by 
(i) the trapezoidal method 
(ii) the cubic spline method. 
In each case, divide the range into two equal subintervals and 
approximate to the solution. Compare your results with the exact 
solution. 
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10.1 INTRODUCTION 


In Chapters 7 and 8 we discussed finite difference methods for the solution 
of boundary-value problems defined by ordinary and partial differential equations. 
We now describe another class of methods for the solution of such problems, 
known as the finite element methods. A full discussion of these methods is 
outside the scope of this book—as normally this does not form part of an 
introductory course on numerical methods. We give here only a brief presentation 
s0 as to enable the reader to know that such methods exist. The discussion 
includes an elementary formulation of the method with simple applications 
to ordinary and partial differential equations. For details, the reader is referred 
to the excellent book by Reddy [1985]. 

The basic idea behind the finite element method is to replace a continuous 
function by means of piecewise polynomials. Such an approximation, called 
the piecewise polynomial approximation, will be discussed in Section 10.1.2. 
The reader is already aware of the importance of polynomial approximations 
in numerical analysis. These are used in the numerical solution of practical 
problems where the exact functions are difficult to obtain or cumbersome 
to use. The idea of piecewise polynomial approximation is also not new to 
the reader, since the cubic spline already discussed, belongs to this class of 
polynomials. 

In engineering applications, several approximate methods of solution are 
used and the reader is familiar with a few of them, e.g. the method of least 
squares, method of collocation, etc, In Section 10.2, we discuss two important 
methods of approximation, viz., the Rayleigh—Ritz method and the Galerkin 


387 
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technique. Rayleigh developed the method to solve certain vibration problems 
and Ritz provided a mathematical basis for it and also applied it to more 
general problems. Whereas the Rayleigh—Ritz method is based on the existence 
of a functional (see Section 10.1.1), the Galerkin technique uses the governing 
equations of the problem and minimizes the error of the approximate solution. 
The latter does not require a functional. A disadvantage of both these 
methods is that higher-order polynomials have to be used to obtain reasonable 
accuracy. 

The finite element method, described in the present chapter, is one of 
the most important numerical applications of the Rayleigh-Ritz and Galerkin 
methods. Its mathematical software is quite popular and used extensively in 
the solution of many practical problems of engineering and applied science. 
In the finite element method, the domain of integration is subdivided into a 
number of smaller regions called elements and over each of these elements 
the continuous function is approximated by a suitable piecewise polynomial. 
To obtain a better approximation one need not use higher-order polynomials 
but only use a finer subdivision, i.e. increase the number of elements. 

In practice, several types of elements are in use, the type used being 
largely dependent upon the geometrical shape of the region under consideration. 
In two-dimensional problems, the elements used are triangles, rectangles 
and quadrilaterals. For three-dimensional problems, tetrahedra, hexahedra 
and parallelopiped elements are used. Since our attempt in this chapter is 
only to introduce the finite element method, we restrict our discussion to 
the use of triangular elements in the solution of simple two-dimensional 
problems (see Section 10.4.2). 

Examples of typical finite elements are shown in Fig. 10.1. 


(a) Line element (b) Triangular and Quadrilateral elements 
Figure 10.1 


10.1.1 Functionals 


The concept of a functional is required to understand the Rayleigh—Ritz 
method, which will be discussed in the next section. This concept arises in 
the study ‘of variational principles, which occur widely in physical and other 
problems. Mathematically, a variational principle consists in determining the 
extreme value of the integral of a typical function, say f(x, y, ¥'). Here the 
integrand is a function of the coordinates and their derivatives and the 
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integration is performed over a region. Consider, for example, the integral 
defined by 


b 
I= | fy. v) ae, (10.1) 


where p(x) satisfies the boundary conditions »(a)= y(5)=0. 

The integrand f is integrated over the one-dimensional domain x, / is 
said to be a functional and is defined as a function which transforms a 
function y into a real number, the value of the definite integral in (10.1). From 
calculus of variations we know that a necessary condition for /(y) to have 
an extremum is that (x) must satisfy the Euler-Lagrange differential equation 


¥_4(2)-o, (10.2)* 
dy de \ Gy 
Similarly, for functionals of the form 
b 
ity) = | Tix yyy ae (10.3) 
a 
the Euler—Lagrange equation takes the form 
x42) £(Z)-0 10.4) 
ey de oy’ wee ay" ‘ ( 


The Euler-Lagrange equation (10.2) has several solutions and the one which 
satisfies the given boundary conditions is selected. Thus, one determines the 
functional so that it takes on an extremum value from a set of permissible 
functions. This is the central problem of a variational principle, An important 
point here is that an extremum may not exist. In other words, a variational 
principle may exist, but an extremum may not exist. Furthermore, not all 
differential equations have a variational principle. These difficulties are serious 
and therefore impose limitations on the application of the variational principle 
to the solution of engineering problems. 

Many problems arising in physics and engineering are modelled by 
boundary-value problems and initial boundary-value problems. Frequently, 
these equations are equivalent to the problem of the minimization of a 
functional which can be interpreted in terms of the total energy of the given 
system. In any physical situation, therefore, the functional is obtained from 
a consideration of the total energy explicitly. Mathematically, however, it 
would be useful to be able to determine the functional from the governing 
differential equation itself. This is illustrated below with an example. 


*For example, see Sastry (1997, a]. 
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Example 10.1 Find the functional for the boundary-value problem defined by 
2 


Ff) (i) 
y(a) = y(b) =0. (ii) 
We have 
b & 
5] fyx=[ fdydx 
a a 


b 

d*y ; d*y 
= { sy dx, since (x)=. 
ye. ' tO 


b 6 
-|25,| -| & © y)dr, on integrating by parts 


a dad ] = = 

= dy (6 y) dx, since 6 y(a) =d y(b) =0 
dy .{ dy | a 
ral Je since 6y)=6f 


il 
I 
a=—SC7 =o 9 =o 
B |S 


bo | 
=) 
‘i 


il 

d, 
5 — 
Ne 
B/S 

e 


b 
1/ dy 
6 += =0. 
j[oes(2) | 
It follows that a unique solution of the problem (i) to (ii) exists at a 
minimum value of the integral defined by 


iy) =f" | wo3( 2) [a (ii) 
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By definition, therefore, the integral in (iii) represents the required functional 
of the problem. In a similar way, functionals of other boundary-value and initial 
boundary-value problems can be derived, 

It is outside the scope of this book to deal extensively with the determination 
of functionals corresponding to boundary-value problems. We list below 
some familiar boundary-value problems with their associated functionals 
and these would be useful in understanding the problems discussed in this 
chapter. 


2 
() <F= 10, ya=y(0)=0 (10.5) 
b 
I(v)= f var -v") ds. (10.6) 
' d*y ; dy 
(il) ai th=* O<x<]; p(0)=0, (2) -1 (10.7) 
i tl(avy 
-_ orl ewe ri ee 
lo)=5 i\(2) kv? + Dux | w(1). (10.8) 
(iif) x*y" + 2xy’ = f(x), y(a) = (5) =0 (10.9) 
b 
I(v)= Jolar-Zeryn |e (10.10) 
(iv) V?u=0, «=0 on the boundary C of 2. (10.11) 
Nv) = If (2 } as ja (10.12) 
(v) V?u=-f, w=0 on the boundary C of RB. (10.13) 
1} (au? (auy 
I(v) = J (2) {%) |-w}4 dy. (10.14) 
(vi) £1? + by f(x), O<x<l 
(10.15) 
*y 
yaO= 7 at x= 0, 
] d*y ' 3 
woes | a(S) + kv? — 2f | de. (10.16) 
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Then eq. (v) becomes 


A nA Al 
Iv)=-2)) ap, - >) >) ae, ay 
i=] i=) j=! 


Hence d//da,; =0 gives 
n 
2p,+2) agy=0, (i=l, 2,....7). (viii) 
j=l 


We wish to find an approximate solution with mn =2 and we therefore choose 
¢,(x)=x(l-x) and ¢)(x)=x*(1-x), so that the boundary conditions (iv) 
are satisfied. 

Now, from (vi), we have 


Also, ¢j(x)=1-2x and ¢5(x)=2x — 3x”, Equation (vii) gives 


1 
| 
ni Be ea 
1 l 
2 =~ | (-2x) 2x-3x7) de =-— = 921, by symmetry 
0 


i 
922 =~ | (Qx~3x") dens, 
; 15 


Equations (vill) now give 
4a, + ary =1 and 10a, + Bar, =i, 
whose solution is a, =a, =1/6. Hence 
fz) = 2x(1-x)+27(1-2)= +x( ost, 
It can be verified that this is the exact solution of the problem (i). 


Example 10.3 Solve the boundary-value problem defined by 
yo+ye-x, O<x<l (i) 
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with 
y(0)= y(1)=0 (ii) 
The exact solution of the problem (i) and (ii) is given by 
sin x 


y(x) = x, (iii) 


sin! 3 


To find the approximate solution by the Rayleigh—Ritz method, we take the 
functional in the form 


] 
Iv) = fo" +7 + 2ux) de. (iv) 
i 


Let an approximate solution be given by 


W(x) = > a(x), (v) 
i=1 
where 
¢,(0) = ¢(1)=0 for all i. (vi) 


Substituting for v in (iv), we obtain 


lj a nA fn A A 
Ioy= []¥ a9) Ya P+ Y aih(x) Yah (x) +2x ¥° 9,2) |e 
0 Li=l i=] i=] 


j=l jal 
(vii) 
As in the previous example, we let 
1 
p= | x(a (vii) 
0 
and 
l r | 
ay = | HCxdoj Cade =— f ogy x)ae (ix) 
0 it] 
Further, let 
| 
ry = | HCDO(x) de. (x) 
0 


Equation (vii) now becomes 


w=) x QO Qi ‘> >} BQ Fy 2> a, p; (xi) 
i=l 


f=] j=l #=1 yl 
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The approximate solution is given by 


(1) gol Le 


124 
The student should compare this with the exact solution. 


10.2.2 The Galerkin Method 


The Rayleigh—Ritz method discussed in Section 10.2.1 is a powerful technique 
for the solution of boundary-value problems. It has, however, the disadvantage 
of requiring the existence of a functional which is not always possible to 
obtain. In fact, not all differential equations have a variational principle. 
Most engineering problems are expressed in terms of certain governing 
equations and boundary conditions, and not in terms of a functional. Galerkin’s 
method belongs to a wider class of methods called the weighted residual 
methods, In this method, an approximating function called the trial fumetion 
(which satisfies all the boundary conditions) is substituted in the given 
differential equation and the result is called the residual (the result will not 
be zero since we have substituted an approximating function). The residual 
is then weighted and the integral of the product, taken over the domain, is 
then set to zero. [t can be shown that if the Euler-Lagrange equation 
corresponding to a functional coincides with the differential equation of the 
problem, then both the Rayleigh—Ritz and Galerkin methods yield the same 
system of equations. 

To explain Galerkin’s method, we consider the boundary value problem 
defined by 


y"+ p(x)y'+q(x)y= f(x), a<x<b (10.27) 
with the boundary conditions 
Pov(a) + goy'(a) = hy 


Pry(b)+ qy (6) = 4 
To find an approximate solution of the problems (10.27) and (10.28), we 
choose base functions ¢,(x) as in the Rayleigh—Ritz method. 


Then an approximate solution v(x) is assumed to be a linear combination 
of the ¢,, ie. v(x) is written as 


(10.28) 


vic) = » a, d(x). (10.29) 


Now, vix) will not, in general, satisfy (10.27), but produces a residual or 
discrepancy. This is equal to the difference between the left-hand and right- 
hand sides of Eq. (10.27) when on the left side p(x) is replaced by v(x). If 
A(v) is the residual, we then write 


Riv) =v" + p(x)v' + g(x)v — f(x). (10.30) 
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Taking the weight function as w,(x), we write 


b 
[ vi) RO) dr =0, (10.31) 
@ 


which yields a system of equations for the unknown parameters a, and can 
be solved. In Galerkin’s method, we usually take y,(x) = ¢,(x). The method 
is illustrated with the following example: 


Example 10.5 We consider again the problem of Example 10.3, viz., 


yl +y=-x, O<x<] (i) 
y(0) = yf) =0 (ii) 

As our first approximation, we choose 
v(x) = a,¢,(x) = a, x(1- x), (iii) 


where ¢,(0) = ¢,(1) =0. 
Substituting for v in (i), we obtain 


Rivj=v"+v4exr (iv) 
Hence, using (10.31), we write 


] 
[o"+v42) A(x) de=0 
0 

l 


[ @" +¥+2)x(1-x) dr =0 (v) 
i] 


Now, 
1 1 I 
|v’ x0-x) de =[v'x(1-2)]5 - | vd-2x) de =~ [ vid-2x) a, 
0 ri 0 
since the first expression on the right vanishes. Now, 
1 1 I 
|v" 2-2) de=-{vl- 2x91} - [-2vde=-2 f vax, since v(0)=v(1)=0. 
o O O 
Hence (v) becomes 


1 
[t-2v+m0-2+70-n]ar=0, (vi) 
0 


which gives on simplification @, = 5/18 = 0.2778. (vii) 
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be a first approximation to w. Clearly, v satisfies the boundary conditions, 
ic. v=0 on the boundary C. The derivatives are given by 

av av 

—=a —1)(2x-1); —=aex(x-D(2y-1); 

ae er er yy 


3 @ 
Bt 72a v-Ds peared. 


Substituting for v in (ii), we obtain 


(iv) 


| 
1) = [ [ axy(e-1) (yD) [2k -2ay(y-1)~2ax(x-]dedy. (v) 
o 0 


eT 
ll 


ay (XDD dy= = 


i =a es : 
xy? (x-D(y-1)? de dy = 7 (vi) 


tl 


o 
Il 


x*y(x-1) G- dr dy=-——. 


Equation (v) now simplifies to 
I(v) = 2kaa -2a*b -2a*e. 


Hence 
al 
—=0= 2ko -—4o0h-—4da 
om ae 
Thus 
ak 5 
= = -—k, j i}. 
tha 4 ee 


It follows that the required approximation for wu is given by 
we v=—2key(x—1 (yD. 


The student should aia that the Galerkin method gives the same solution 
as above. 


10.4 THE FINITE ELEMENT METHOD 
The Rayleigh-Ritz and Galerkin methods, discussed in the previous sections, 


cannot be applied directly for obtaining the global approximate solutions of 
engineering problems. An important reason for this is the difficulty associated 
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Section 10.4 : The Finite Element Method 407 
Instead of Eq. (10.40), we now have 
K fy!) a re), (10.50) 


where K}*) and F;) are given by (10.41) and (10.42). 

With ‘the choice of d(x) as in Eq. (10.48), we now demonstrate the 
computation of K“*) and Fe), In particular, we choose a(x)=1 and f =2. 
With A, =x, —x,_;, we obtain 


eal ani mee (10.51) 
z dx oh, 
where 
2 
l Il 
c= }{-tfaot 
eh ae he 
1 i 
Ky = | ~pa den = Ka | (10.52) 
i i 
a 
| l 
Ka = | — ar a_ 
2 
| he he 
and 
ag ety 
FO =2 | “a + Di =h, + Di") 
= (10,53) 
ay 
X~NXp_ 
Ff =2 | Side py =h, +D®, 
Xe=] 
As a particular case, we consider the following example. 
Example 10.7 We consider the following problem defined by 
d* ¥ : 
——-=-2, 0<xr<l, y(0)=0, ¥'()=0. (1) 
dx” 
The exact solution of the above problem is given by 
p(x) = 2x27 (ii) 


Comparison with (10.32) shows that a(x)=1 and f(x)=2. 
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(a) To demonstrate the steps involved in the finite element solution, we 
divide [0,1] into two equal subintervals with f, =1/2. From (10.48) and 
(10.49), we obtain the equations for both elements. 


(i) e=1: x, =0, x, =1/2, 


2 —2 typ 
KO FO el 
2 2 5+ Dy 
Gi) e¢=2: x,_, =1/2, x, =1 
2 ~2 14D 
K@— , Fas? 
~2 2 5+D?? 


Having determined the equations for each element, these have to be assembled 
now to determine the global approximations. This will be the next step in 
the finite element solution. 

Step 4 (Assembly of element equations): We shall explain this step with 
reference to the two elements obtained in Example 10.7. In this case, the 
two elements are connected at the node 2. Since the function (x) is continuous, 
it follows that y, of element 1 should be the same as y, of element 2. For 
the two elements of Example 10.7, the correspondence can be expressed 
mathematically as follows: 


| 2 Z 
yak, gp anay, yh ak, 


In the finite element analysis, such relations are usually called interelement 
continuity conditions. 

Using the above relations, the global finite element model of the given 
boundary value problem is 


2 =-2 Oo}; ¥ V/2+D") 
2 2420-2]! Bla/14 D9 +d |. 
0 =) 2|| % V2+D?) 
The next step is the imposition of boundary conditions. 


Step 5 (Imposition af boundary conditions): The homogeneous boundary 
condition gives ¥, = 0. Then, we obtain the equations: 


4%-2%=1, -2% +2%, =5 
since DS" and D{” cancel each other and D{?) = 0 is the natural boundary 
condition. The solution of this system is given by 
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To avoid confusion, we now write down the complete system for each element 


4 4 0 0 9} AY jiv4+Df 
~4 4 0 oO Ol] 1/44 DW 
e=! 0 0 0 0 Of] Kl=| gy 
0 oOo 0 o of] ¥ 0 
0 oOo 0 0 Of] & 0 
0 oOo o oO olf ¥ 0 
0 4 -+4 oO Off %| |14+D) 
e=72 0) —4 4 0 0 |= 44 p? 
o Oo oOo 0 o]F% 0 
0 oo o o ol] & 0 
0 0 o oO Of ¥ 0 
o 0 oOo oOo of % 0 
e=3 |0 0 4 -4~ Off %]=|/4+D> 
0 0 -4 4 Off %&| |1/4+D2) 
o 0 oo Oo oO F% 0 
0 o oO 0 ov; HT f oa 
0 0 0 90 0|| % 0 
e=4 0 0 0 Oo olf KI=| ° 
pi) 
0 0 0 4 -4|1 % I 
0 0 0 + 4|| ¥,| | DS? 
Adding up the above, we obtain 
4 “a 0 0 OK] i444 D0 
4 444 -4 0 Ol yt fia+DM +d 
0 ~4 444 4 Olly, |=] 1/2+ DP?) + D/? 
0 0 4 444-4) y | | 1/2+D2) + Di 
0 0 0 4 4] ¥ | |is+D$" 


By boundary condition, we have ¥, =0. 


Presented By: http://www.ebooksuit.com 


Hidden page 


Presented By: http://www.ebooksuit.com 


412 Cuaprer 10: The Finite Element Method 


To find the variational form of Eq. (10.54), we multiply it with the test 
function v(x, y) and integrate the result over a typical element R,, to obtain 


ff FE+F 3) yao fone (10.56) 


where 


ou 
In = Ty ae +iy ay’ 
}, and 1, being the direction cosines of a unit normal 4 on the boundary 
¢, and ds is an arc length of an infinitesimal element along the boundary. 
The next step in the finite element solution of this problem is to set up 
a finite element model of the given equation. To do this, we approximate wu 
by the expression 


vey ud; (10.57) 
j=l 
where u, =u(x,, y,) and the ¢, have the property 
l, if i=/ 
Aes )=5){ 0 fie]. (10.58) 


Substituting (10.57) in (10.56) and putting v=¢,, we obtain 


a9, a6, 24, 
0 ie: t+ 7 7A a de dy— J] sadeay- | ance 059) 


for i=l, 2, ...,m. 
Equation (10.59) can be written in the form 


3 Ku) =F (10.60) 
jel 
where 
(e) _ 3, OF) 8g, 26; 
K‘ ii(2 a By |e (10.61) 
and 
FO = fj fb, de dy+ J qn, ds. (10.62) 
Re c 


Equation (10.60) represents the finite element model of the Poisson equation. 
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We next consider a triangular element (see Fig. 10.3) in which the 
nodes are numbered in the counter-clockwise direction and derive the 
interpolation functions for it. We assume the interpolating polynomial in 
such a way that the number of terms in it equals the number of nodes in 
the triangular element. Accordingly, we assume 


u(x, Y)=a, +a,x+ ayy (10.63) 


1%, 4) 


0 x 
Figure 10,3 
as the required approximation. We also set 
u(x;, yj) =U, i=l, 2, 3 (10.64) 


where (x,, ¥,), /=1, 2, 3 denote the three vertices of the triangle. Substituting 
(10.64) in (10.63), we obtain 


Wy = a + apX) + 45) 
My =, + G9X5 + Gy)5 (10.65) 


My © Oh} + ag Xy + 05); 
Solving Eqs. (10.65), we obtain 


; uy uy uy 
ae 3A. x x7 ¥3 
M1 ¥2 ¥3 
\ Mm \ MY My 
cree BL | ¥2 ¥3 (10.66) 
2h, l 1 l 
1 uy uz Uy 
a | x4 x3 
where 
i es 
A, = Area of the triangle = = 1 ox yy = 10.67) 


[xy yy 
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Substituting for a), a3, a; in (10.63) and simplifying, we obtain 


l 
2A, 


w(x, ¥) = —— [my (x9 %3 — X32) + Hg (4) — 1.9) + 5 (4 2 — 2.4) 


l 
hh [my C2 — ¥3) + a (Vy — + 3(H — Yd) 
or [uy (x3 — x) + Mg (ay — 43) +05 (%y -— x) Ly. (10.68) 


Collecting the coefficients of u,, uz and w; in the above, Eq. (10.68) can 
be written in the form 


3 
u(x, y)= x M, g(x, yh (10.69) 


i=] 


where the ¢ ) are the linear interpolating functions for the triangular elements 
under consideration, and are given by 


' l x y 
g(x, ara 1 x Wh 
A, 
1 8 OY 
l x ¥ 
(e) 2a 
¢; (x, A ava l 4 (10.70) 
Le ee 
I x ¥ 
g(x, Y=—]1 4 HI, 
1 mm 


From formulae (10.70), it is easily verified that 


1, if i=/ 


3 
d te) = 10.71 
0 ifiey * L9G y= uP 


f=] 


Hany )={ 


We also have 


(e) (e) 
o#3 Sas = (10.72) 
a 24, ' a 2A, 
a4) y-» Of) os 
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Using Eqs. (10.72) the element matrices K{*) and F;) in (10.60) can then 
be computed easily. These computations will be demonstrated through a 


simple example. 


Example 10.8 We consider a particular case of the problem defined by 
Eqs. (10.54) and (10.55), viz., the Poisson equation 


{23+23)-2 O<x, ysl é (i) 


with the condition 
u=0 (ii) 
on the boundary of the square 0<x<1, 0s ysl. 

We divide the square region along the line of symmetry * = ¥ and then 
consider only the lower-triangular part. We again subdivide the lower triangular 
part into four triangular elements, as shown in Fig. 10.4. Let the elements 
be numbered, as shown in the figure, and it is seen that element @,@ and 
@ are symmetrical. Hence the element matrices for these elements will all 
be of the same type. 


(0, 1) 


0 2 (1, 0) 


Figure 10.4 


Now, the vertices 1, 2 and 3 of the element @ are given by (0,0), (1/2, 0) 
and (1/2, 1/2) respectively. For this element, we obtain A, =1/8 and 


Eqs. (10.70) give 
# a3 o4( 5 | l-2x 


4 =4|04 52-5 y|=26-y) (ii) 


gf =4[o+ty]=2y 
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It is easy to see that ¢{ + ¢{ + 9 =1, thus verifying (10.71). The element 
matrices K and F can now be computed easily, using (10.61) and (10.62). 
We first obtain the derivatives 


(i) ag!) 
aan 7 
(I) a) 
a) ta -2 (iv) 
(I) 
a wo, 6S | 
dy 
Equation (10.61) now gives 


Kip =f j 4d dy ==, KQ) = [[-4dedy=-2, Ki) =0 
a Fe) 


kK) =-<, KQ=1, KY =-2 (v) 


1 l 1 
KY) =0, Ki => KS = 


Similarly Eq. (10.62) yields 
FO =f 2(1—2x) dx dy + J q,(1—2x) ds 
&y23 C3 


W2 V2 
x j [ 2 -2x) de dy +1{ say 
o 6 


= +1, where I{P = [| g,(1-2x) ds 


Cis 
F=f f 22x-2y)dedy+ | 9,(2x-2y)ds (vi) 
A123 23 
=< +1), where 10 = fg, (2x-2y) ds 
Chas 
F=f f tydedys | gul2y)as 
Ann Cia 


aot, where If) = fg, (2y)ds 


Cin 
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Let the global nodes be Uj, U;, Uy, Uy, Us, and Us, corresponding to the 
local nodes wy, 4, M3, My, Ws and ws at the respective vertices. As there are 
six nodes, the corresponding matrices will be of order 6. Hence, we obtain 
for element @: 


2-2 0 0 0 0 1 | jaf? 
-1/2 1 -1/2 0 O 0 } 1) 
xe 0-2 V2 0 0 Of] ltt |, ro 
0 0 oO 0 0 0 2101 Io 
0 0 o 0 0 O 0 | jo 
0 0 o 0 6 0 0 0 

(vii) 


Since the elements @ and @ are similar to @, their element matrices will 
be of the same type as those of @ given in (vii). Thus, for element @, 


0 0 o oO 0 Oo 0 0 
Oo Ww 0-12 +O 06 i 12) 
0 0 0 0 O 
@). : and Fe) = i 0 + 0 
0 -2 0 1 -I/2 0 ran 12) 
0 0 oO -I/2 1/2 0 I i 
0 o oO 0 0 oO 0 0 
(viii) 
Similarly, for element @, 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0); |0 
i) 
ei 0 0 V2 0 -2 Of} pl t ; 
0 0 0 0 0 0 lz 0 
1 | | 
0 oOo -1I/72 0 1 -i/2 2 
{4} 
o 0 oOo oO -W2 12 i ie 


(ix) 
Finally, for element @, we note that the correspondence between its vertices 
and those of @ is given by 51,342, and 2-3. Hence, we have 
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0 0 0 o o oO 07 fo 
0 W2 -2 O 0 O S 
0-2 1 oO -1/2 0 1] }i 
K® = and FO) + 
0 oO 0 0 0 0 l2) 0 0 
0 0 -12 O 2 © 1) [a 
0 0 0 o oO 0} Io 


(x) 
Assembling the element matrices in (vii), (viii), (ix) and (x) and simplifying, 
we obtain the matrix equation 


1 -1 oO 9 0 olf uy, eo 
| F | = =| 0 0 U; 3 he nT a +10) 
\ o -2 4 0 2 0 77a ed te / @ a 12) Pe, 4) 
= =_—<— de 
2) 0 -l 0 z -| Ol] uy) 12) 4 i @) 
0 0 0 0 -+ af U% 13” 
(xi) 
From the boundary conditions, we have (see Fig. 10.4) 
uy =, =U, =U, =(/, =0. (xii) 
Hence, eq. (xi) gives 
—U, =i 41? +149 +12) (xiii) 
] : 
2U, rhe: i +19) +1{9 (xiv) 
l 4 
—U; math er + ) (xv) 
From (xiv), we obtain 
1 | : 
U; =etsds +19 +1) (xvi) 
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d*y , dy 

1 ‘ —— —_— = = — 7 “ 

0.3. x7 + 2x g(x), y(0)= y(1)=0 
d? ¥ 

10.4. —> + p(x) y+9(x)=0, y(a)= y(b)=0. 


2 


10.5. dy y= f(x) O<x<l, yoo 2 <0 ax=0,1. 


ax' ax” 
10.6. V7~=0, w=0 on the boundary C of R. 
10.7. V?u=-f, u=0 on the boundary C of R. 


Use the Rayleigh—Ritz method to solve the following boundary-value problems 
(Problems 8-11): 


d*y 

10.8. oak y(0) = p(l) = 0. 
d* ¥ 

10.9. = +y=x', y(0)= y(1) =0. (Use a two-parameter approximate 
solution). 


a 
10.10, Fz tx Zt y=2x, WO)=1 and (0) =0. 


ri 
10.11. SF +x2-2y=0, y'(0) + (0) =1 and y(1) =2. 


Compare the two-parameter approximate solution with the analytical solution 
given by y=x* +1. Apply the Galerkin technique to solve the following 
boundary-value problems (Problems 12-14): 

10.12. Exercise 9 above. 

10.13. Exercise 11 above. 


d? , 1 
10.14. eye =(0) 0 =4 ‘1 -_—=, 
7~F=% W=8 YO=-3 


10.15. Using Galerkin technique, solve Poisson’s equation 
ee Q<x, y<l 
ax? ay? 
with w=0 on the boundary C of the region R. 


10.16. Use the Galerkin technique to approximate Eg. (10.36) and hence 
obtain the solution of the boundary-value problem defined by 
d’y 
—-=—2, O<x<1; O=0, y'(D=0, 
ae y(0) ¥W 


taking two equal subintervals. 
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10.17. In the notation of Example 10.8, prove that (a) 1°) =0 (b) /{® =0. 
10.18. (Reddy). Solve Poisson's equation 


{2 Oru 


—-+— }=l, O<x, yl, 
ax? | 


where 


= ou = =u(x = 
Beebe 0; ull, yeux, D=0. 
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